Compare commits

...
Author SHA1 Message Date
dependabot[bot]anddependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> 2acec9c0dc Bump @types/node from 17.0.45 to 18.0.2 (#140)
Bumps [@types/node](https://github.com/DefinitelyTyped/DefinitelyTyped/tree/HEAD/types/node) from 17.0.45 to 18.0.2.
- [Release notes](https://github.com/DefinitelyTyped/DefinitelyTyped/releases)
- [Commits](https://github.com/DefinitelyTyped/DefinitelyTyped/commits/HEAD/types/node)

---
updated-dependencies:
- dependency-name: "@types/node"
  dependency-type: direct:development
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2022-07-06 19:04:46 +02:00
Jimmy Wärting 20ee08f33e feat: new way of creating temporary files backed up by fs (#137)
* feat: new way of creating temporary files backed up by fs

* add readme
2022-07-06 19:01:17 +02:00
Jimmy Wärting f077471a23 fix: avoid reading files when the size is empty (#134)
* fix: avoid reading files when the size is empty

* also use stat.size for checking if it is modified

* bump minor version number

* Don't push empty chunks into the array (creates faster reads)

* reverted back to avoid making a if check
2022-03-10 15:11:10 +01:00
Jimmy Wärting dc29588366 fix: misc updates (#132)
* misc updates

* bring back c8 for test coverage

* use new loader

* don't use engines

* simplify ci

* fix double slice problem
2022-01-20 14:00:15 +01:00
Jimmy Wärting 8ab587d340 removed xo 2021-10-30 13:52:13 +02:00
Jimmy Wärting f61ee01955 bumped minor patch 2021-10-30 13:50:21 +02:00
Jimmy Wärting 5dbcbb91b4 standard.js style 2021-10-30 13:50:21 +02:00
Jimmy Wärting c5c2b52154 fix(warning): Supress warnings when importing experimental web stream from NodeJS (#125)
* Suppress warnings when importing web stream
* added node: prefix
* updated changelog
* apply `standard streams.cjs --fix`
2021-10-30 13:04:02 +02:00
Ben McCann 836709fc65 fix: prefix stream/web import with node: (#122) 2021-10-24 17:27:58 +02:00
Nick K 2c11647222 fix(blob, file, tests): Minor improvements for Blob and File implementations (#120)
* Use private class fields in parts initialization directly.

* Move encoder initialization outside the loop

* Improve errors reported by Blob constructor.

* Simulate WebIDL type casting for lastModified in File constructor.

* Add cancel method for Blob.stream() underlying source.

* Fix indentation inconsistency.

* Use getReader in blob stream cancellation test instead of Symbol.asyncIterator
2021-10-24 14:56:15 +02:00
Jimmy Wärting 9022f8fc70 Update index.js 2021-09-29 17:14:07 +02:00
Jimmy Wärting 0e07457246 Added v12 to test 2021-09-07 17:16:24 +02:00
14 changed files with 858 additions and 921 deletions
+5 -29
View File
@@ -14,42 +14,18 @@ jobs:
strategy:
matrix:
os: [ubuntu-latest, windows-latest, macOS-latest]
node: ["16", "15", "14", engines]
exclude:
# On Windows, run tests with only the LTS environments.
- os: windows-latest
node: engines
- os: windows-latest
node: "14"
# On macOS, run tests with only the LTS environments.
- os: macOS-latest
node: engines
- os: macOS-latest
node: "14"
node: ["17.3"]
runs-on: ${{ matrix.os }}
steps:
- uses: actions/checkout@v2
- name: Get Node.JS version from package.json
if: matrix.node == 'engines'
id: get-version
run: echo ::set-output name=node::$(npx --q minimum-node-version)
- uses: actions/setup-node@v2-beta
if: matrix.node != 'engines'
- uses: actions/setup-node@v2
with:
node-version: ${{ matrix.node }}
- uses: actions/setup-node@v2-beta
if: matrix.node == 'engines'
with:
node-version: ${{steps.get-version.outputs.node}}
node-version: '17.3'
- run: npm install
- run: npm test
- run: npm run report -- --colors
- name: Upload coverage to Codecov
uses: codecov/codecov-action@v1
with:
-20
View File
@@ -1,20 +0,0 @@
name: CI
on:
pull_request:
paths:
- "**.js"
- "**eslint**"
- "package.json"
- ".github/workflows/lint.yml"
jobs:
lint:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v2
- name: Use Node.js
uses: actions/setup-node@v2-beta
with:
node-version: 14
- run: npm install
- run: npm run lint
+6 -2
View File
@@ -16,6 +16,8 @@ pids
*.seed
*.pid.lock
# Directory for instrumented libs generated by jscoverage/JSCover
lib-cov
@@ -47,8 +49,8 @@ typings/
# Optional npm cache directory
.npm
# Optional eslint cache
.eslintcache
# Optional cache
.cache
# Optional REPL history
.node_repl_history
@@ -63,3 +65,5 @@ typings/
.env
*.d.ts
*.d.cts
.DS_Store
+5 -10
View File
@@ -5,22 +5,16 @@ All notable changes to this project will be documented in this file.
The format is based on [Keep a Changelog](http://keepachangelog.com/) and this
project adheres to [Semantic Versioning](http://semver.org/).
(Unreleased)
==================
### Removed
-
### Changed
-
### Added
-
### Fixed
## v3.1.3
- Allow usage of iterable object in Blob constructor. [#108]
- Run test WPT test against our impl [#109]
- File name are now casted to string [#109]
- Slicing in the middle of multiple parts added more bytes than what what it should have [#109]
- Prefixed `stream/web` import with `node:` to allow easier static analysis detection of Node built-ins [#122]
- Added `node:` prefix in `from.js` as well [#114]
- Suppress warning when importing `stream/web` [#114]
## v3.1.2
- Improved typing
- Fixed a bug where position in iterator did not increase
@@ -101,3 +95,4 @@ project adheres to [Semantic Versioning](http://semver.org/).
[#108]: https://github.com/node-fetch/fetch-blob/pull/108
[#109]: https://github.com/node-fetch/fetch-blob/pull/109
[#114]: https://github.com/node-fetch/fetch-blob/pull/114
+30
View File
@@ -87,6 +87,34 @@ console.log(blob.size) // ~4 GiB
`blobFrom|blobFromSync|fileFrom|fileFromSync(path, [mimetype])`
### Creating a temporary file on the disk
(requires [FinalizationRegistry] - node v14.6)
When using both `createTemporaryBlob` and `createTemporaryFile`
then you will write data to the temporary folder in their respective OS.
The arguments can be anything that [fsPromises.writeFile] supports. NodeJS
v14.17.0+ also supports writing (async)Iterable streams and passing in a
AbortSignal, so both NodeJS stream and whatwg streams are supported. When the
file have been written it will return a Blob/File handle with a references to
this temporary location on the disk. When you no longer have a references to
this Blob/File anymore and it have been GC then it will automatically be deleted.
This files are also unlinked upon exiting the process.
```js
import { createTemporaryBlob, createTemporaryFile } from 'fetch-blob/from.js'
const req = new Request('https://httpbin.org/image/png')
const res = await fetch(req)
const type = res.headers.get('content-type')
const signal = req.signal
let blob = await createTemporaryBlob(res.body, { type, signal })
// const file = createTemporaryBlob(res.body, 'img.png', { type, signal })
blob = undefined // loosing references will delete the file from disk
```
`createTemporaryBlob(data, { type, signal })`
`createTemporaryFile(data, FileName, { type, signal, lastModified })`
### Creating Blobs backed up by other async sources
Our Blob & File class are more generic then any other polyfills in the way that it can accept any blob look-a-like item
An example of this is that our blob implementation can be constructed with parts coming from [BlobDataItem](https://github.com/node-fetch/fetch-blob/blob/8ef89adad40d255a3bbd55cf38b88597c1cd5480/from.js#L32) (aka a filepath) or from [buffer.Blob](https://nodejs.org/api/buffer.html#buffer_new_buffer_blob_sources_options), It dose not have to implement all the methods - just enough that it can be read/understood by our Blob implementation. The minium requirements is that it has `Symbol.toStringTag`, `size`, `slice()` and either a `stream()` or a `arrayBuffer()` method. If you then wrap it in our Blob or File `new Blob([blobDataItem])` then you get all of the other methods that should be implemented in a blob or file
@@ -104,3 +132,5 @@ See the [MDN documentation](https://developer.mozilla.org/en-US/docs/Web/API/Blo
[install-size-image]: https://flat.badgen.net/packagephobia/install/fetch-blob
[install-size-url]: https://packagephobia.now.sh/result?p=fetch-blob
[fs-blobs]: https://github.com/nodejs/node/issues/37340
[fsPromises.writeFile]: https://nodejs.org/dist/latest-v18.x/docs/api/fs.html#fspromiseswritefilefile-data-options
[FinalizationRegistry]: https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/FinalizationRegistry
+27 -18
View File
@@ -1,40 +1,49 @@
import Blob from './index.js';
import Blob from './index.js'
const _File = class File extends Blob {
#lastModified = 0;
#name = '';
#lastModified = 0
#name = ''
/**
* @param {*[]} fileBits
* @param {string} fileName
* @param {{lastModified?: number, type?: string}} options
*/// @ts-ignore
constructor(fileBits, fileName, options = {}) {
constructor (fileBits, fileName, options = {}) {
if (arguments.length < 2) {
throw new TypeError(`Failed to construct 'File': 2 arguments required, but only ${arguments.length} present.`);
throw new TypeError(`Failed to construct 'File': 2 arguments required, but only ${arguments.length} present.`)
}
super(fileBits, options);
super(fileBits, options)
if (options === null) options = {};
if (options === null) options = {}
const modified = Number(options.lastModified);
this.#lastModified = Number.isNaN(modified) ? Date.now() : modified
this.#name = String(fileName);
// Simulate WebIDL type casting for NaN value in lastModified option.
const lastModified = options.lastModified === undefined ? Date.now() : Number(options.lastModified)
if (!Number.isNaN(lastModified)) {
this.#lastModified = lastModified
}
this.#name = String(fileName)
}
get name() {
return this.#name;
get name () {
return this.#name
}
get lastModified() {
return this.#lastModified;
get lastModified () {
return this.#lastModified
}
get [Symbol.toStringTag]() {
return "File";
get [Symbol.toStringTag] () {
return 'File'
}
static [Symbol.hasInstance] (object) {
return !!object && object instanceof Blob &&
/^(File)$/.test(object[Symbol.toStringTag])
}
}
/** @type {typeof globalThis.File} */// @ts-ignore
export const File = _File;
export default File;
export const File = _File
export default File
+131 -72
View File
@@ -1,58 +1,98 @@
import {statSync, createReadStream, promises as fs} from 'fs';
import {basename} from 'path';
import {MessageChannel} from 'worker_threads';
import {
realpathSync,
statSync,
rmdirSync,
createReadStream,
promises as fs
} from 'node:fs'
import { basename, sep, join } from 'node:path'
import { tmpdir } from 'node:os'
import process from 'node:process'
import DOMException from 'node-domexception'
import File from './file.js';
import Blob from './index.js';
import File from './file.js'
import Blob from './index.js'
const {stat} = fs;
const DOMException = globalThis.DOMException || (() => {
const port = new MessageChannel().port1
const ab = new ArrayBuffer(0)
try { port.postMessage(ab, [ab, ab]) }
catch (err) { return err.constructor }
})()
const { stat, mkdtemp } = fs
let i = 0, tempDir, registry
/**
* @param {string} path filepath on the disk
* @param {string} [type] mimetype to use
*/
const blobFromSync = (path, type) => fromBlob(statSync(path), path, type);
const blobFromSync = (path, type) => fromBlob(statSync(path), path, type)
/**
* @param {string} path filepath on the disk
* @param {string} [type] mimetype to use
* @returns {Promise<Blob>}
*/
const blobFrom = (path, type) => stat(path).then(stat => fromBlob(stat, path, type))
/**
* @param {string} path filepath on the disk
* @param {string} [type] mimetype to use
* @returns {Promise<File>}
*/
const fileFrom = (path, type) => stat(path).then(stat => fromFile(stat, path, type))
/**
* @param {string} path filepath on the disk
* @param {string} [type] mimetype to use
*/
const blobFrom = (path, type) => stat(path).then(stat => fromBlob(stat, path, type));
/**
* @param {string} path filepath on the disk
* @param {string} [type] mimetype to use
*/
const fileFrom = (path, type) => stat(path).then(stat => fromFile(stat, path, type));
/**
* @param {string} path filepath on the disk
* @param {string} [type] mimetype to use
*/
const fileFromSync = (path, type) => fromFile(statSync(path), path, type);
const fileFromSync = (path, type) => fromFile(statSync(path), path, type)
// @ts-ignore
const fromBlob = (stat, path, type = '') => new Blob([new BlobDataItem({
path,
size: stat.size,
lastModified: stat.mtimeMs,
start: 0
})], {type});
path,
size: stat.size,
lastModified: stat.mtimeMs,
start: 0
})], { type })
// @ts-ignore
const fromFile = (stat, path, type = '') => new File([new BlobDataItem({
path,
size: stat.size,
lastModified: stat.mtimeMs,
start: 0
})], basename(path), { type, lastModified: stat.mtimeMs });
path,
size: stat.size,
lastModified: stat.mtimeMs,
start: 0
})], basename(path), { type, lastModified: stat.mtimeMs })
/**
* Creates a temporary blob backed by the filesystem.
* NOTE: requires node.js v14 or higher to use FinalizationRegistry
*
* @param {*} data Same as fs.writeFile data
* @param {BlobPropertyBag & {signal?: AbortSignal}} options
* @param {AbortSignal} [signal] in case you wish to cancel the write operation
* @returns {Promise<Blob>}
*/
const createTemporaryBlob = async (data, {signal, type} = {}) => {
registry = registry || new FinalizationRegistry(fs.unlink)
tempDir = tempDir || await mkdtemp(realpathSync(tmpdir()) + sep)
const id = `${i++}`
const destination = join(tempDir, id)
if (data instanceof ArrayBuffer) data = new Uint8Array(data)
await fs.writeFile(destination, data, { signal })
const blob = await blobFrom(destination, type)
registry.register(blob, destination)
return blob
}
/**
* Creates a temporary File backed by the filesystem.
* Pretty much the same as constructing a new File(data, name, options)
*
* NOTE: requires node.js v14 or higher to use FinalizationRegistry
* @param {*} data
* @param {string} name
* @param {FilePropertyBag & {signal?: AbortSignal}} opts
* @returns {Promise<File>}
*/
const createTemporaryFile = async (data, name, opts) => {
const blob = await createTemporaryBlob(data)
return new File([blob], name, opts)
}
/**
* This is a blob backed up by a file on the disk
@@ -62,44 +102,63 @@ const fromFile = (stat, path, type = '') => new File([new BlobDataItem({
* @private
*/
class BlobDataItem {
#path;
#start;
#path
#start
constructor(options) {
this.#path = options.path;
this.#start = options.start;
this.size = options.size;
this.lastModified = options.lastModified
}
constructor (options) {
this.#path = options.path
this.#start = options.start
this.size = options.size
this.lastModified = options.lastModified
this.originalSize = options.originalSize === undefined
? options.size
: options.originalSize
}
/**
* Slicing arguments is first validated and formatted
* to not be out of range by Blob.prototype.slice
*/
slice(start, end) {
return new BlobDataItem({
path: this.#path,
lastModified: this.lastModified,
size: end - start,
start
});
}
/**
* Slicing arguments is first validated and formatted
* to not be out of range by Blob.prototype.slice
*/
slice (start, end) {
return new BlobDataItem({
path: this.#path,
lastModified: this.lastModified,
originalSize: this.originalSize,
size: end - start,
start: this.#start + start
})
}
async * stream() {
const {mtimeMs} = await stat(this.#path)
if (mtimeMs > this.lastModified) {
throw new DOMException('The requested file could not be read, typically due to permission problems that have occurred after a reference to a file was acquired.', 'NotReadableError');
}
yield * createReadStream(this.#path, {
start: this.#start,
end: this.#start + this.size - 1
});
}
async * stream () {
const { mtimeMs, size } = await stat(this.#path)
get [Symbol.toStringTag]() {
return 'Blob';
}
if (mtimeMs > this.lastModified || this.originalSize !== size) {
throw new DOMException('The requested file could not be read, typically due to permission problems that have occurred after a reference to a file was acquired.', 'NotReadableError')
}
yield * createReadStream(this.#path, {
start: this.#start,
end: this.#start + this.size - 1
})
}
get [Symbol.toStringTag] () {
return 'Blob'
}
}
export default blobFromSync;
export {File, Blob, blobFrom, blobFromSync, fileFrom, fileFromSync};
process.once('exit', () => {
tempDir && rmdirSync(tempDir, { recursive: true })
})
export default blobFromSync
export {
Blob,
blobFrom,
blobFromSync,
createTemporaryBlob,
File,
fileFrom,
fileFromSync,
createTemporaryFile
}
+213 -205
View File
@@ -1,246 +1,254 @@
/*! fetch-blob. MIT License. Jimmy Wärting <https://jimmy.warting.se/opensource> */
// TODO (jimmywarting): in the feature use conditional loading with top level await (requires 14.x)
// Node has recently added whatwg stream into core
import './streams.cjs';
/** @typedef {import('buffer').Blob} NodeBlob} */
import './streams.cjs'
// 64 KiB (same size chrome slice theirs blob into Uint8array's)
const POOL_SIZE = 65536;
const POOL_SIZE = 65536
/** @param {(Blob | NodeBlob | Uint8Array)[]} parts */
/** @param {(Blob | Uint8Array)[]} parts */
async function * toIterator (parts, clone = true) {
for (let part of parts) {
if ('stream' in part) {
yield * part.stream();
} else if (ArrayBuffer.isView(part)) {
if (clone) {
let position = part.byteOffset;
let end = part.byteOffset + part.byteLength;
while (position !== end) {
const size = Math.min(end - position, POOL_SIZE);
const chunk = part.buffer.slice(position, position + size);
position += chunk.byteLength;
yield new Uint8Array(chunk);
}
} else {
yield part;
}
} else {
/* c8 ignore start */
// For blobs that have arrayBuffer but no stream method (nodes buffer.Blob)
let position = 0;
while (position !== part.size) {
const chunk = part.slice(position, Math.min(part.size, position + POOL_SIZE));
const buffer = await chunk.arrayBuffer();
position += buffer.byteLength;
yield new Uint8Array(buffer);
}
/* c8 ignore end */
}
}
for (const part of parts) {
if ('stream' in part) {
yield * (/** @type {AsyncIterableIterator<Uint8Array>} */ (part.stream()))
} else if (ArrayBuffer.isView(part)) {
if (clone) {
let position = part.byteOffset
const end = part.byteOffset + part.byteLength
while (position !== end) {
const size = Math.min(end - position, POOL_SIZE)
const chunk = part.buffer.slice(position, position + size)
position += chunk.byteLength
yield new Uint8Array(chunk)
}
} else {
yield part
}
/* c8 ignore next 10 */
} else {
// For blobs that have arrayBuffer but no stream method (nodes buffer.Blob)
let position = 0, b = (/** @type {Blob} */ (part))
while (position !== b.size) {
const chunk = b.slice(position, Math.min(b.size, position + POOL_SIZE))
const buffer = await chunk.arrayBuffer()
position += buffer.byteLength
yield new Uint8Array(buffer)
}
}
}
}
const _Blob = class Blob {
/** @type {Array.<(Blob|Uint8Array)>} */
#parts = []
#type = ''
#size = 0
#endings = 'transparent'
/** @type {Array.<(Blob|Uint8Array)>} */
#parts = [];
#type = '';
#size = 0;
/**
* The Blob() constructor returns a new Blob object. The content
* of the blob consists of the concatenation of the values given
* in the parameter array.
*
* @param {*} blobParts
* @param {{ type?: string, endings?: string }} [options]
*/
constructor (blobParts = [], options = {}) {
if (typeof blobParts !== 'object' || blobParts === null) {
throw new TypeError('Failed to construct \'Blob\': The provided value cannot be converted to a sequence.')
}
/**
* The Blob() constructor returns a new Blob object. The content
* of the blob consists of the concatenation of the values given
* in the parameter array.
*
* @param {*} blobParts
* @param {{ type?: string }} [options]
*/
constructor(blobParts = [], options = {}) {
const parts = [];
let size = 0;
if (typeof blobParts !== 'object') {
throw new TypeError(`Failed to construct 'Blob': parameter 1 is not an iterable object.`);
}
if (typeof blobParts[Symbol.iterator] !== 'function') {
throw new TypeError('Failed to construct \'Blob\': The object must have a callable @@iterator property.')
}
if (typeof options !== 'object' && typeof options !== 'function') {
throw new TypeError(`Failed to construct 'Blob': parameter 2 cannot convert to dictionary.`);
}
if (typeof options !== 'object' && typeof options !== 'function') {
throw new TypeError('Failed to construct \'Blob\': parameter 2 cannot convert to dictionary.')
}
if (options === null) options = {};
if (options === null) options = {}
for (const element of blobParts) {
let part;
if (ArrayBuffer.isView(element)) {
part = new Uint8Array(element.buffer.slice(element.byteOffset, element.byteOffset + element.byteLength));
} else if (element instanceof ArrayBuffer) {
part = new Uint8Array(element.slice(0));
} else if (element instanceof Blob) {
part = element;
} else {
part = new TextEncoder().encode(element);
}
const encoder = new TextEncoder()
for (const element of blobParts) {
let part
if (ArrayBuffer.isView(element)) {
part = new Uint8Array(element.buffer.slice(element.byteOffset, element.byteOffset + element.byteLength))
} else if (element instanceof ArrayBuffer) {
part = new Uint8Array(element.slice(0))
} else if (element instanceof Blob) {
part = element
} else {
part = encoder.encode(`${element}`)
}
size += ArrayBuffer.isView(part) ? part.byteLength : part.size;
parts.push(part);
}
const size = ArrayBuffer.isView(part) ? part.byteLength : part.size
// Avoid pushing empty parts into the array to better GC them
if (size) {
this.#size += size
this.#parts.push(part)
}
}
const type = options.type === undefined ? '' : String(options.type);
this.#endings = `${options.endings === undefined ? 'transparent' : options.endings}`
const type = options.type === undefined ? '' : String(options.type)
this.#type = /^[\x20-\x7E]*$/.test(type) ? type : ''
}
this.#type = /^[\x20-\x7E]*$/.test(type) ? type : '';
this.#size = size;
this.#parts = parts;
}
/**
* The Blob interface's size property returns the
* size of the Blob in bytes.
*/
get size () {
return this.#size
}
/**
* The Blob interface's size property returns the
* size of the Blob in bytes.
*/
get size() {
return this.#size;
}
/**
* The type property of a Blob object returns the MIME type of the file.
*/
get type () {
return this.#type
}
/**
* The type property of a Blob object returns the MIME type of the file.
*/
get type() {
return this.#type;
}
/**
* The text() method in the Blob interface returns a Promise
* that resolves with a string containing the contents of
* the blob, interpreted as UTF-8.
*
* @return {Promise<string>}
*/
async text () {
// More optimized than using this.arrayBuffer()
// that requires twice as much ram
const decoder = new TextDecoder()
let str = ''
for await (const part of toIterator(this.#parts, false)) {
str += decoder.decode(part, { stream: true })
}
// Remaining
str += decoder.decode()
return str
}
/**
* The text() method in the Blob interface returns a Promise
* that resolves with a string containing the contents of
* the blob, interpreted as UTF-8.
*
* @return {Promise<string>}
*/
async text() {
// More optimized than using this.arrayBuffer()
// that requires twice as much ram
const decoder = new TextDecoder();
let str = '';
for await (let part of toIterator(this.#parts, false)) {
str += decoder.decode(part, { stream: true });
}
// Remaining
str += decoder.decode();
return str;
}
/**
* The arrayBuffer() method in the Blob interface returns a
* Promise that resolves with the contents of the blob as
* binary data contained in an ArrayBuffer.
*
* @return {Promise<ArrayBuffer>}
*/
async arrayBuffer () {
// Easier way... Just a unnecessary overhead
// const view = new Uint8Array(this.size);
// await this.stream().getReader({mode: 'byob'}).read(view);
// return view.buffer;
/**
* The arrayBuffer() method in the Blob interface returns a
* Promise that resolves with the contents of the blob as
* binary data contained in an ArrayBuffer.
*
* @return {Promise<ArrayBuffer>}
*/
async arrayBuffer() {
// Easier way... Just a unnecessary overhead
// const view = new Uint8Array(this.size);
// await this.stream().getReader({mode: 'byob'}).read(view);
// return view.buffer;
const data = new Uint8Array(this.size)
let offset = 0
for await (const chunk of toIterator(this.#parts, false)) {
data.set(chunk, offset)
offset += chunk.length
}
const data = new Uint8Array(this.size);
let offset = 0;
for await (const chunk of toIterator(this.#parts, false)) {
data.set(chunk, offset);
offset += chunk.length;
}
return data.buffer
}
return data.buffer;
}
stream () {
const it = toIterator(this.#parts, true)
stream() {
const it = toIterator(this.#parts, true);
return new globalThis.ReadableStream({
// @ts-ignore
type: 'bytes',
async pull (ctrl) {
const chunk = await it.next()
chunk.done ? ctrl.close() : ctrl.enqueue(chunk.value)
},
return new ReadableStream({
type: 'bytes',
async pull(ctrl) {
const chunk = await it.next();
chunk.done ? ctrl.close() : ctrl.enqueue(chunk.value);
}
})
}
async cancel () {
await it.return()
}
})
}
/**
* The Blob interface's slice() method creates and returns a
* new Blob object which contains data from a subset of the
* blob on which it's called.
*
* @param {number} [start]
* @param {number} [end]
* @param {string} [type]
*/
slice(start = 0, end = this.size, type = '') {
const {size} = this;
/**
* The Blob interface's slice() method creates and returns a
* new Blob object which contains data from a subset of the
* blob on which it's called.
*
* @param {number} [start]
* @param {number} [end]
* @param {string} [type]
*/
slice (start = 0, end = this.size, type = '') {
const { size } = this
let relativeStart = start < 0 ? Math.max(size + start, 0) : Math.min(start, size);
let relativeEnd = end < 0 ? Math.max(size + end, 0) : Math.min(end, size);
let relativeStart = start < 0 ? Math.max(size + start, 0) : Math.min(start, size)
let relativeEnd = end < 0 ? Math.max(size + end, 0) : Math.min(end, size)
const span = Math.max(relativeEnd - relativeStart, 0);
const parts = this.#parts;
const blobParts = [];
let added = 0;
const span = Math.max(relativeEnd - relativeStart, 0)
const parts = this.#parts
const blobParts = []
let added = 0
for (const part of parts) {
// don't add the overflow to new blobParts
if (added >= span) {
break;
}
for (const part of parts) {
// don't add the overflow to new blobParts
if (added >= span) {
break
}
const size = ArrayBuffer.isView(part) ? part.byteLength : part.size;
if (relativeStart && size <= relativeStart) {
// Skip the beginning and change the relative
// start & end position as we skip the unwanted parts
relativeStart -= size;
relativeEnd -= size;
} else {
let chunk
if (ArrayBuffer.isView(part)) {
chunk = part.subarray(relativeStart, Math.min(size, relativeEnd));
added += chunk.byteLength
} else {
chunk = part.slice(relativeStart, Math.min(size, relativeEnd));
added += chunk.size
}
relativeEnd -= size;
blobParts.push(chunk);
relativeStart = 0; // All next sequential parts should start at 0
}
}
const size = ArrayBuffer.isView(part) ? part.byteLength : part.size
if (relativeStart && size <= relativeStart) {
// Skip the beginning and change the relative
// start & end position as we skip the unwanted parts
relativeStart -= size
relativeEnd -= size
} else {
let chunk
if (ArrayBuffer.isView(part)) {
chunk = part.subarray(relativeStart, Math.min(size, relativeEnd))
added += chunk.byteLength
} else {
chunk = part.slice(relativeStart, Math.min(size, relativeEnd))
added += chunk.size
}
relativeEnd -= size
blobParts.push(chunk)
relativeStart = 0 // All next sequential parts should start at 0
}
}
const blob = new Blob([], {type: String(type).toLowerCase()});
blob.#size = span;
blob.#parts = blobParts;
const blob = new Blob([], { type: String(type).toLowerCase() })
blob.#size = span
blob.#parts = blobParts
return blob;
}
return blob
}
get [Symbol.toStringTag]() {
return 'Blob';
}
get [Symbol.toStringTag] () {
return 'Blob'
}
static [Symbol.hasInstance](object) {
return (
object &&
typeof object === 'object' &&
typeof object.constructor === 'function' &&
(
typeof object.stream === 'function' ||
typeof object.arrayBuffer === 'function'
) &&
/^(Blob|File)$/.test(object[Symbol.toStringTag])
);
}
static [Symbol.hasInstance] (object) {
return (
object &&
typeof object === 'object' &&
typeof object.constructor === 'function' &&
(
typeof object.stream === 'function' ||
typeof object.arrayBuffer === 'function'
) &&
/^(Blob|File)$/.test(object[Symbol.toStringTag])
)
}
}
Object.defineProperties(_Blob.prototype, {
size: {enumerable: true},
type: {enumerable: true},
slice: {enumerable: true}
});
size: { enumerable: true },
type: { enumerable: true },
slice: { enumerable: true }
})
/** @type {typeof globalThis.Blob} */
export const Blob = _Blob;
export default Blob;
export const Blob = _Blob
export default Blob
+8 -36
View File
@@ -1,6 +1,6 @@
{
"name": "fetch-blob",
"version": "3.1.2",
"version": "3.1.5",
"description": "Blob & File implementation in Node.js, originally from node-fetch.",
"main": "index.js",
"type": "module",
@@ -14,11 +14,9 @@
"streams.cjs"
],
"scripts": {
"lint": "xo test.js",
"test-wpt": "node --experimental-loader ./test/http-loader.js ./test/test-wpt-in-node.js",
"test": "npm run lint && ava test.js",
"report": "c8 --reporter json --reporter text ava test.js",
"coverage": "c8 --reporter json --reporter text ava test.js && codecov -f coverage/coverage-final.json",
"test": "node --experimental-loader ./test/http-loader.js ./test/test-wpt-in-node.js",
"report": "c8 --reporter json --reporter text npm run test",
"coverage": "npm run report && codecov -f coverage/coverage-final.json",
"prepublishOnly": "tsc --declaration --emitDeclarationOnly --allowJs index.js from.js"
},
"repository": "https://github.com/node-fetch/fetch-blob.git",
@@ -36,37 +34,10 @@
"url": "https://github.com/node-fetch/fetch-blob/issues"
},
"homepage": "https://github.com/node-fetch/fetch-blob#readme",
"xo": {
"rules": {
"unicorn/prefer-node-protocol": "off",
"unicorn/numeric-separators-style": "off",
"unicorn/prefer-spread": "off",
"unicorn/prefer-number-properties": "off",
"import/extensions": [
"error",
"always",
{
"ignorePackages": true
}
]
},
"overrides": [
{
"files": "test.js",
"rules": {
"node/no-unsupported-features/es-syntax": 0,
"node/no-unsupported-features/node-builtins": 0
}
}
]
},
"devDependencies": {
"ava": "^3.15.0",
"c8": "^7.7.2",
"codecov": "^3.8.2",
"node-fetch": "^3.0.0-beta.9",
"typescript": "^4.3.2",
"xo": "^0.40.1"
"@types/node": "^18.0.2",
"c8": "^7.11.0",
"typescript": "^4.5.4"
},
"funding": [
{
@@ -79,6 +50,7 @@
}
],
"dependencies": {
"node-domexception": "^1.0.0",
"web-streams-polyfill": "^3.0.3"
}
}
+35 -21
View File
@@ -1,37 +1,51 @@
/* c8 ignore start */
// 64 KiB (same size chrome slice theirs blob into Uint8array's)
const POOL_SIZE = 65536;
const POOL_SIZE = 65536
if (!globalThis.ReadableStream) {
// `node:stream/web` got introduced in v16.5.0 as experimental
// and it's preferred over the polyfilled version. So we also
// suppress the warning that gets emitted by NodeJS for using it.
try {
Object.assign(globalThis, require('stream/web'))
const process = require('node:process')
const { emitWarning } = process
try {
process.emitWarning = () => {}
Object.assign(globalThis, require('node:stream/web'))
process.emitWarning = emitWarning
} catch (error) {
process.emitWarning = emitWarning
throw error
}
} catch (error) {
// TODO: Remove when only supporting node >= 16.5.0
// fallback to polyfill implementation
Object.assign(globalThis, require('web-streams-polyfill/dist/ponyfill.es2018.js'))
}
}
try {
const {Blob} = require('buffer')
// Don't use node: prefix for this, require+node: is not supported until node v14.14
// Only `import()` can use prefix in 12.20 and later
const { Blob } = require('buffer')
if (Blob && !Blob.prototype.stream) {
Blob.prototype.stream = function name(params) {
let position = 0;
const blob = this;
Blob.prototype.stream = function name (params) {
let position = 0
const blob = this
return new ReadableStream({
type: 'bytes',
async pull(ctrl) {
const chunk = blob.slice(position, Math.min(blob.size, position + POOL_SIZE));
const buffer = await chunk.arrayBuffer();
position += buffer.byteLength;
ctrl.enqueue(new Uint8Array(buffer))
return new ReadableStream({
type: 'bytes',
async pull (ctrl) {
const chunk = blob.slice(position, Math.min(blob.size, position + POOL_SIZE))
const buffer = await chunk.arrayBuffer()
position += buffer.byteLength
ctrl.enqueue(new Uint8Array(buffer))
if (position === blob.size) {
ctrl.close()
}
}
})
}
}
if (position === blob.size) {
ctrl.close()
}
}
})
}
}
} catch (error) {}
/* c8 ignore end */
-396
View File
@@ -1,396 +0,0 @@
import fs from 'fs';
import {Readable} from 'stream';
import buffer from 'buffer';
import test from 'ava';
import {Response} from 'node-fetch';
import syncBlob, {blobFromSync, blobFrom, fileFromSync, fileFrom} from './from.js';
import File from './file.js';
import Blob from './index.js';
const license = fs.readFileSync('./LICENSE', 'utf-8');
test('new Blob()', t => {
const blob = new Blob(); // eslint-disable-line no-unused-vars
t.pass();
});
test('new Blob(parts)', t => {
const data = 'a=1';
const blob = new Blob([data]); // eslint-disable-line no-unused-vars
t.pass();
});
test('Blob ctor parts', async t => {
const parts = [
'a',
new Uint8Array([98]),
new Uint16Array([25699]),
new Uint8Array([101]).buffer,
Buffer.from('f'),
new Blob(['g']),
{},
new URLSearchParams('foo')
];
const blob = new Blob(parts);
t.is(await blob.text(), 'abcdefg[object Object]foo=');
});
test('Blob ctor threats an object with @@iterator as a sequence', async t => {
const blob = new Blob({[Symbol.iterator]: Array.prototype[Symbol.iterator]});
t.is(blob.size, 0);
t.is(await blob.text(), '');
});
test('Blob ctor reads blob parts from object with @@iterator', async t => {
const input = ['one', 'two', 'three'];
const expected = input.join('');
const blob = new Blob({
* [Symbol.iterator]() {
yield * input;
}
});
t.is(blob.size, new TextEncoder().encode(expected).byteLength);
t.is(await blob.text(), expected);
});
test('Blob ctor throws a string', t => {
t.throws(() => new Blob('abc'));
});
test('Blob ctor threats Uint8Array as a sequence', async t => {
const input = [1, 2, 3];
const blob = new Blob(new Uint8Array(input));
t.is(await blob.text(), input.join(''));
});
test('Blob size', t => {
const data = 'a=1';
const blob = new Blob([data]);
t.is(blob.size, data.length);
});
test('Blob type', t => {
const type = 'text/plain';
const blob = new Blob([], {type});
t.is(blob.type, type);
});
test('Blob slice type', t => {
const type = 'text/plain';
const blob = new Blob().slice(0, 0, type);
t.is(blob.type, type);
});
test('invalid Blob type', t => {
const blob = new Blob([], {type: '\u001Ftext/plain'});
t.is(blob.type, '');
});
test('invalid Blob slice type', t => {
const blob = new Blob().slice(0, 0, '\u001Ftext/plain');
t.is(blob.type, '');
});
test('Blob text()', async t => {
const data = 'a=1';
const type = 'text/plain';
const blob = new Blob([data], {type});
t.is(await blob.text(), data);
});
test('Blob arrayBuffer()', async t => {
const data = 'a=1';
const type = 'text/plain';
const blob = new Blob([data], {type});
const decoder = new TextDecoder('utf-8');
const buffer = await blob.arrayBuffer();
t.is(decoder.decode(buffer), data);
});
test('Blob stream()', async t => {
const data = 'a=1';
const type = 'text/plain';
const blob = new Blob([data], {type});
for await (const chunk of blob.stream()) {
t.is(chunk.join(), [97, 61, 49].join());
}
});
test('Blob toString()', t => {
const data = 'a=1';
const type = 'text/plain';
const blob = new Blob([data], {type});
t.is(blob.toString(), '[object Blob]');
});
test('Blob slice()', async t => {
const data = 'abcdefgh';
const blob = new Blob([data]).slice();
t.is(await blob.text(), data);
});
test('Blob slice(0, 1)', async t => {
const data = 'abcdefgh';
const blob = new Blob([data]).slice(0, 1);
t.is(await blob.text(), 'a');
});
test('Blob slice(-1)', async t => {
const data = 'abcdefgh';
const blob = new Blob([data]).slice(-1);
t.is(await blob.text(), 'h');
});
test('Blob slice(0, -1)', async t => {
const data = 'abcdefgh';
const blob = new Blob([data]).slice(0, -1);
t.is(await blob.text(), 'abcdefg');
});
test('Blob(["hello ", "world"]).slice(5)', async t => {
const parts = ['hello ', 'world'];
const blob = new Blob(parts);
t.is(await blob.slice(5).text(), ' world');
});
test('throw away unwanted parts', async t => {
const blob = new Blob(['a', 'b', 'c']).slice(1, 2);
t.is(await blob.text(), 'b');
});
test('Blob works with node-fetch Response.blob()', async t => {
const data = 'a=1';
const type = 'text/plain';
const blob = new Blob([data], {type});
const response = new Response(Readable.from(blob.stream()));
const blob2 = await response.blob();
t.is(await blob2.text(), data);
});
test('Blob works with node-fetch Response.text()', async t => {
const data = 'a=1';
const type = 'text/plain';
const blob = new Blob([data], {type});
const response = new Response(Readable.from(blob.stream()));
const text = await response.text();
t.is(text, data);
});
test('blob part backed up by filesystem', async t => {
const blob = blobFromSync('./LICENSE');
t.is(await blob.slice(0, 3).text(), license.slice(0, 3));
t.is(await blob.slice(4, 11).text(), license.slice(4, 11));
});
test('Reading after modified should fail', async t => {
const blob = blobFromSync('./LICENSE');
await new Promise(resolve => {
setTimeout(resolve, 500);
});
fs.closeSync(fs.openSync('./LICENSE', 'a'));
const error = await t.throwsAsync(blob.text());
t.is(error.constructor.name, 'DOMException');
t.is(error instanceof Error, true);
t.is(error.name, 'NotReadableError');
const file = fileFromSync('./LICENSE');
// Above test updates the last modified date to now
t.is(typeof file.lastModified, 'number');
// The lastModifiedDate is deprecated and removed from spec
t.false('lastModifiedDate' in file);
const mod = file.lastModified - Date.now();
t.true(mod <= 0 && mod >= -500); // Close to tolerance: 0.500ms
});
test('Reading file after modified should fail', async t => {
const file = fileFromSync('./LICENSE');
await new Promise(resolve => {
setTimeout(resolve, 100);
});
const now = new Date();
// Change modified time
fs.utimesSync('./LICENSE', now, now);
const error = await t.throwsAsync(file.text());
t.is(error.constructor.name, 'DOMException');
t.is(error instanceof Error, true);
t.is(error.name, 'NotReadableError');
});
test('Reading from the stream created by blobFrom', async t => {
const blob = blobFromSync('./LICENSE');
const actual = await blob.text();
t.is(actual, license);
});
test('create a blob from path asynchronous', async t => {
const blob = await blobFrom('./LICENSE');
const actual = await blob.text();
t.is(actual, license);
});
test('Reading empty blobs', async t => {
const blob = blobFromSync('./LICENSE').slice(0, 0);
const actual = await blob.text();
t.is(actual, '');
});
test('Blob-ish class is an instance of Blob', t => {
class File {
stream() {}
get [Symbol.toStringTag]() {
return 'File';
}
}
t.true(new File() instanceof Blob);
});
test('Instanceof check returns false for nullish values', t => {
t.false(null instanceof Blob);
});
/** @see https://github.com/w3c/FileAPI/issues/43 - important to keep boundary value */
test('Dose not lowercase the blob values', t => {
const type = 'multipart/form-data; boundary=----WebKitFormBoundaryTKqdrVt01qOBltBd';
t.is(new Blob([], {type}).type, type);
});
test('Parts are immutable', async t => {
const buf = new Uint8Array([97]);
const blob = new Blob([buf]);
buf[0] = 98;
t.is(await blob.text(), 'a');
});
test('Blobs are immutable', async t => {
const buf = new Uint8Array([97]);
const blob = new Blob([buf]);
const chunk = await blob.stream().getReader().read();
t.is(chunk.value[0], 97);
chunk.value[0] = 98;
t.is(await blob.text(), 'a');
});
// This was necessary to avoid large ArrayBuffer clones (slice)
test('Large chunks are divided into smaller chunks', async t => {
const buf = new Uint8Array(65590);
const blob = new Blob([buf]);
let i = 0;
// eslint-disable-next-line no-unused-vars
for await (const chunk of blob.stream()) {
i++;
}
t.is(i === 2, true);
});
test('Can use named import - as well as default', async t => {
// eslint-disable-next-line node/no-unsupported-features/es-syntax
const {Blob, default: def} = await import('./index.js');
t.is(Blob, def);
});
test('default from.js exports blobFromSync', t => {
t.is(blobFromSync, syncBlob);
});
if (buffer.Blob) {
test('Can wrap buffer.Blob to a fetch-blob', async t => {
const blob1 = new buffer.Blob(['blob part']);
const blob2 = new Blob([blob1]);
t.is(await blob2.text(), 'blob part');
});
}
test('File is a instance of blob', t => {
t.true(new File([], '') instanceof Blob);
});
test('fileFrom returns the name', async t => {
t.is((await fileFrom('./LICENSE')).name, 'LICENSE');
});
test('fileFromSync returns the name', t => {
t.is(fileFromSync('./LICENSE').name, 'LICENSE');
});
test('fileFromSync(path, type) sets the type', t => {
t.is(fileFromSync('./LICENSE', 'text/plain').type, 'text/plain');
});
test('blobFromSync(path, type) sets the type', t => {
t.is(blobFromSync('./LICENSE', 'text/plain').type, 'text/plain');
});
test('fileFrom(path, type) sets the type', async t => {
const file = await fileFrom('./LICENSE', 'text/plain');
t.is(file.type, 'text/plain');
});
test('new File(,,{lastModified: 100})', t => {
const mod = new File([], '', {lastModified: 100}).lastModified;
t.is(mod, 100);
});
test('new File(,,{lastModified: "200"})', t => {
const mod = new File([], '', {lastModified: '200'}).lastModified;
t.is(mod, 200);
});
test('new File(,,{lastModified: true})', t => {
const mod = new File([], '', {lastModified: true}).lastModified;
t.is(mod, 1);
});
test('new File(,,{lastModified: new Date()})', t => {
const mod = new File([], '', {lastModified: new Date()}).lastModified - Date.now();
t.true(mod <= 0 && mod >= -20); // Close to tolerance: 0.020ms
});
test('new File(,,{}) sets current time', t => {
const mod = new File([], '').lastModified - Date.now();
t.true(mod <= 0 && mod >= -20); // Close to tolerance: 0.020ms
});
test('blobFrom(path, type) sets the type', async t => {
const blob = await blobFrom('./LICENSE', 'text/plain');
t.is(blob.type, 'text/plain');
});
test('blobFrom(path) sets empty type', async t => {
const blob = await blobFrom('./LICENSE');
t.is(blob.type, '');
});
test('new File() throws with too few args', t => {
t.throws(() => new File(), {
instanceOf: TypeError,
message: 'Failed to construct \'File\': 2 arguments required, but only 0 present.'
});
});
test('can slice zero sized blobs', async t => {
const blob = new Blob();
const txt = await blob.slice(0, 0).text();
t.is(txt, '');
});
test('returns a readable stream', t => {
const stream = new File([], '').stream();
t.true(typeof stream.getReader === 'function');
});
test('checking instanceof blob#stream', async t => {
// eslint-disable-next-line node/no-unsupported-features/es-syntax
const {ReadableStream} = await import('stream/web').catch(_ => import('web-streams-polyfill/dist/ponyfill.es2018.js'));
const stream = new File([], '').stream();
t.true(stream instanceof ReadableStream);
});
+42 -28
View File
@@ -1,8 +1,21 @@
// https-loader.mjs
import { get } from 'https';
import { Buffer } from 'node:buffer'
import fs from 'node:fs'
import { get } from 'node:https'
export function resolve(specifier, context, defaultResolve) {
const { parentURL = null } = context;
const fetch = url => new Promise(rs => get(url, rs))
const cache = new URL('./.cache/', import.meta.url)
/**
* @param {string} specifier
* @param {{
* conditions: !Array<string>,
* parentURL: !(string | undefined),
* }} context
* @param {Function} defaultResolve
* @returns {Promise<{ url: string }>}
*/
export async function resolve (specifier, context, defaultResolve) {
const { parentURL = null } = context
// Normally Node.js would error on specifiers starting with 'https://', so
// this hook intercepts them and converts them into absolute URLs to be
@@ -10,42 +23,43 @@ export function resolve(specifier, context, defaultResolve) {
if (specifier.startsWith('https://')) {
return {
url: specifier
};
}
} else if (parentURL && parentURL.startsWith('https://')) {
return {
url: new URL(specifier, parentURL).href
};
}
}
// Let Node.js handle all other specifiers.
return defaultResolve(specifier, context, defaultResolve);
return defaultResolve(specifier, context, defaultResolve)
}
export function getFormat(url, context, defaultGetFormat) {
// This loader assumes all network-provided JavaScript is ES module code.
if (url.startsWith('https://')) {
return {
format: 'module'
};
}
// Let Node.js handle all other URLs.
return defaultGetFormat(url, context, defaultGetFormat);
}
export function getSource(url, context, defaultGetSource) {
export async function load (url, context, defaultLoad) {
// For JavaScript to be loaded over the network, we need to fetch and
// return it.
if (url.startsWith('https://')) {
return new Promise((resolve, reject) => {
let data = ''
get(url, async res => {
for await (const chunk of res) data += chunk;
resolve({ source: data });
}).on('error', (err) => reject(err));
});
const uuid = Buffer.from(url).toString('hex')
const cachedFile = new URL(uuid, cache)
let data = ''
// cache remote files for 1h
if (fs.existsSync(cachedFile) && fs.statSync(cachedFile).mtimeMs > Date.now() - 1000 * 60 * 60) {
data = fs.readFileSync(cachedFile, 'utf8')
} else {
const res = await fetch(url).catch(err => err)
for await (const chunk of res) data += chunk
fs.mkdirSync(cache, { recursive: true })
fs.writeFileSync(cachedFile, data)
}
return {
// This example assumes all network-provided JavaScript is ES module
// code.
format: 'module',
source: data
}
}
// Let Node.js handle all other URLs.
return defaultGetSource(url, context, defaultGetSource);
return defaultLoad(url, context, defaultLoad)
}
+262
View File
@@ -0,0 +1,262 @@
// @ts-nocheck
// @ts-ignore
import fs from 'node:fs'
import buffer from 'node:buffer'
import syncBlob, {
blobFromSync,
blobFrom,
fileFromSync,
fileFrom,
createTemporaryBlob,
createTemporaryFile
} from '../from.js'
const license = fs.readFileSync('./LICENSE')
test_blob(() => new Blob([
'a',
new Uint8Array([98]),
new Uint16Array([25699]),
new Uint8Array([101]).buffer,
Buffer.from('f'),
new Blob(['g']),
{},
new URLSearchParams('foo')
]), {
desc: 'Blob ctor parts',
expected: 'abcdefg[object Object]foo=',
type: '',
length: 26
})
promise_test(async () => {
assert_equals(fileFromSync('./LICENSE').name, 'LICENSE')
assert_equals((await fileFrom('./LICENSE')).name, 'LICENSE')
}, 'file from returns the same name')
// Could not find similar test on WPT
test(() => {
const now = new Date()
assert_equals(new File([], '', { lastModified: now }).lastModified, +now)
assert_equals(new File([], '', { lastModified: +now }).lastModified, +now)
assert_equals(new File([], '', { lastModified: 100 }).lastModified, 100)
assert_equals(new File([], '', { lastModified: '200' }).lastModified, 200)
assert_equals(new File([], '', { lastModified: true }).lastModified, 1)
assert_equals(new File([], '', { lastModified: false }).lastModified, 0)
assert_approx_equals(new File([], '').lastModified, Date.now(), 0.020)
assert_approx_equals(new File([], '', { lastModified: undefined }).lastModified, Date.now(), 0.020)
}, 'File sets current time')
// Could not find similar test on WPT
test(() => {
const values = ['Not a Number', [], {}, null]
// I can't really see anything about this in the spec,
// but this is how browsers handle type casting for this option...
for (const lastModified of values) {
const file = new File([], '', { lastModified })
assert_equals(file.lastModified, 0)
}
}, 'Interpretes NaN value in lastModified option as 0')
test(() => {
assert_equals(blobFromSync, syncBlob)
}, 'default export is named exported blobFromSync')
promise_test(async () => {
const { Blob, default: def } = await import('../index.js')
assert_equals(Blob, def)
}, 'Can use named import - as well as default')
// This was necessary to avoid large ArrayBuffer clones (slice)
promise_test(async t => {
const buf = new Uint8Array(65590)
const blob = new Blob([buf])
let i = 0
// eslint-disable-next-line no-unused-vars
for await (const chunk of blob.stream()) {
i++
}
assert_equals(i, 2)
}, 'Large chunks are divided into smaller chunks')
// Couldn't find a test for this on WPT
promise_test(async () => {
const buf = new Uint8Array([97])
const blob = new Blob([buf])
const chunk = await blob.stream().getReader().read()
assert_equals(chunk.value[0], 97)
chunk.value[0] = 98
assert_equals(await blob.text(), 'a')
}, 'Blobs are immutable')
/**
* Deviation from WPT: it's important to keep boundary value
* so we don't lowercase the type
* @see https://github.com/w3c/FileAPI/issues/43
*/
test(() => {
const type = 'multipart/form-data; boundary=----WebKitFormBoundaryTKqdrVt01qOBltBd'
assert_equals(new Blob([], { type }).type, type)
assert_equals(new File([], '', { type }).type, type)
}, 'Dose not lowercase the type')
test( // Because we have symbol.hasInstance it's best to test it...
() => (assert_false(null instanceof Blob), assert_false(null instanceof File)),
'Instanceof check returns false for nullish values'
)
test( // Because browser normally never tries things taken for granted
() => assert_equals(new Blob().toString(), '[object Blob]'),
'blob.toString() returns [object Blob]'
)
test( // Because browser normally never tries things taken for granted
() => assert_equals(new File([], '').toString(), '[object File]'),
'file.toString() returns [object File]'
)
// fetch-blob uniques is that it supports arbitrary blobs too
test(() => {
class File {
stream () {}
get [Symbol.toStringTag] () { return 'File' }
}
assert_true(new File() instanceof Blob)
}, 'Blob-ish class is an instance of Blob')
// fetch-blob uniques is that it supports arbitrary blobs too
if (buffer.Blob) {
test_blob(() => new Blob([new buffer.Blob(['blob part'])]), {
desc: 'Can wrap buffer.Blob to a fetch-blob',
expected: 'blob part',
type: '',
})
}
/**
* Test if Blob can be constructed with BOM and keep it when casted to string
* Test if blob.text() can correctly remove BOM - `buffer.toString()` is bad
*/
promise_test(async () => {
const text = '{"foo": "bar"}'
const blob = new Blob([`\uFEFF${text}`])
assert_equals(blob.size, 17)
assert_equals(await blob.text(), text)
const ab = await blob.slice(0, 3).arrayBuffer()
assert_equals_typed_array(new Uint8Array(ab), new Uint8Array([0xEF, 0xBB, 0xBF]))
}, 'Can wrap buffer.Blob to a fetch-blob')
// Here to make sure our `toIterator` is working as intended
promise_test(async () => {
const stream = new Blob(['Some content']).stream()
// Cancel the stream before start reading, or this will throw an error
await stream.cancel()
const reader = stream.getReader()
const { done, value: chunk } = await reader.read()
assert_true(done)
assert_equals(chunk, undefined)
}, 'Blob stream() can be cancelled')
/******************************************************************************/
/* */
/* Test Blobs backed up by the filesystem */
/* */
/******************************************************************************/
promise_test(async () => {
assert_equals(fileFromSync('./LICENSE', 'text/plain').type, 'text/plain')
assert_equals(fileFromSync('./LICENSE').type, '')
assert_equals(blobFromSync('./LICENSE', 'text/plain').type, 'text/plain')
assert_equals(blobFromSync('./LICENSE').type, '')
assert_equals((await fileFrom('./LICENSE', 'text/plain')).type, 'text/plain')
assert_equals((await fileFrom('./LICENSE')).type, '')
assert_equals((await blobFrom('./LICENSE', 'text/plain')).type, 'text/plain')
assert_equals((await blobFrom('./LICENSE')).type, '')
}, 'from utilities sets correct type')
promise_test(async () => {
assert_equals(await blobFromSync('./LICENSE').text(), license.toString())
assert_equals(await fileFromSync('./LICENSE').text(), license.toString())
assert_equals(await (await blobFrom('./LICENSE')).text(), license.toString())
assert_equals(await (await fileFrom('./LICENSE')).text(), license.toString())
}, 'blob part backed up by filesystem can be read')
promise_test(async () => {
assert_equals(await blobFromSync('./LICENSE').text(), license.toString())
assert_equals(await fileFromSync('./LICENSE').text(), license.toString())
assert_equals(await (await blobFrom('./LICENSE')).text(), license.toString())
assert_equals(await (await fileFrom('./LICENSE')).text(), license.toString())
}, 'blob part backed up by filesystem slice correctly')
promise_test(async () => {
let blob
// Can construct a temporary blob from a string
blob = await createTemporaryBlob(license.toString())
assert_equals(await blob.text(), license.toString())
// Can construct a temporary blob from a async iterator
blob = await createTemporaryBlob(blob.stream())
assert_equals(await blob.text(), license.toString())
// Can construct a temporary file from a arrayBuffer
blob = await createTemporaryBlob(await blob.arrayBuffer())
assert_equals(await blob.text(), license.toString())
// Can construct a temporary file from a arrayBufferView
blob = await createTemporaryBlob(await blob.arrayBuffer().then(ab => new Uint8Array(ab)))
assert_equals(await blob.text(), license.toString())
// Can specify a mime type
blob = await createTemporaryBlob('abc', { type: 'text/plain' })
assert_equals(blob.type, 'text/plain')
// Can create files too
let file = await createTemporaryFile('abc', 'abc.txt', {
type: 'text/plain',
lastModified: 123
})
assert_equals(file.name, 'abc.txt')
assert_equals(file.size, 3)
assert_equals(file.lastModified, 123)
}, 'creating temporary blob/file backed up by filesystem')
promise_test(async () => {
fs.writeFileSync('temp', '')
await blobFromSync('./temp').text()
fs.unlinkSync('./temp')
}, 'can read empty files')
test(async () => {
const blob = blobFromSync('./LICENSE')
await new Promise(resolve => setTimeout(resolve, 2000))
const now = new Date()
// Change modified time
fs.utimesSync('./LICENSE', now, now)
const error = await blob.text().then(assert_unreached, e => e)
assert_equals(error.constructor.name, 'DOMException')
assert_equals(error instanceof Error, true)
assert_equals(error.name, 'NotReadableError')
const file = fileFromSync('./LICENSE')
// Above test updates the last modified date to now
assert_equals(typeof file.lastModified, 'number')
// The lastModifiedDate is deprecated and removed from spec
assert_false('lastModifiedDate' in file)
assert_approx_equals(file.lastModified, +now, 1000)
}, 'Reading after modified should fail')
promise_test(async () => {
assert_equals(await blobFromSync('./LICENSE').slice(0, 0).text(), '')
assert_equals(await blobFromSync('./LICENSE').slice(0, 3).text(), license.slice(0, 3).toString())
assert_equals(await blobFromSync('./LICENSE').slice(4, 11).text(), license.slice(4, 11).toString())
assert_equals(await blobFromSync('./LICENSE').slice(-11).text(), license.slice(-11).toString())
assert_equals(await blobFromSync('./LICENSE').slice(4, 11).slice(2, 5).text(), license.slice(4, 11).slice(2, 5).toString())
}, 'slicing blobs backed up by filesystem returns correct string')
+94 -84
View File
@@ -1,133 +1,143 @@
// Don't want to use the FileReader, don't want to lowerCase the type either
// import from 'https://wpt.live/resources/testharnessreport.js'
import {File, Blob} from '../from.js'
import { File, Blob } from '../from.js'
let hasFailed
globalThis.self = globalThis
await import('https://wpt.live/resources/testharness.js')
// Should probably be fixed... should be able to compare a Blob to a File
delete Blob[Symbol.hasInstance]
setup({
explicit_timeout: true,
explicit_done: true,
});
explicit_done: true
})
function test_blob(fn, expectations) {
var expected = expectations.expected,
type = expectations.type,
desc = expectations.desc;
function test_blob (fn, expectations) {
const expected = expectations.expected
const type = expectations.type
const desc = expectations.desc
const length = expectations.length
var t = async_test(desc);
t.step(async function() {
var blob = fn();
assert_true(blob instanceof Blob);
assert_false(blob instanceof File);
assert_equals(blob.type.toLowerCase(), type);
assert_equals(blob.size, expected.length);
assert_equals(await blob.text(), expected);
t.done();
});
const t = async_test(desc)
t.step(async function () {
const blob = fn()
assert_true(blob instanceof Blob)
assert_false(blob instanceof File)
assert_equals(blob.type.toLowerCase(), type)
assert_equals(await blob.text(), expected)
t.done()
})
}
function test_blob_binary(fn, expectations) {
var expected = expectations.expected,
type = expectations.type,
desc = expectations.desc;
function test_blob_binary (fn, expectations) {
const expected = expectations.expected
const type = expectations.type
const desc = expectations.desc
var t = async_test(desc);
t.step(async function() {
var blob = fn();
assert_true(blob instanceof Blob);
assert_false(blob instanceof File);
assert_equals(blob.type.toLowerCase(), type);
assert_equals(blob.size, expected.length);
const result = await blob.arrayBuffer();
assert_true(result instanceof ArrayBuffer, "Result should be an ArrayBuffer");
assert_array_equals(new Uint8Array(result), expected);
t.done();
});
const t = async_test(desc)
t.step(async function () {
const blob = fn()
assert_true(blob instanceof Blob)
assert_false(blob instanceof File)
assert_equals(blob.type.toLowerCase(), type)
const result = await blob.arrayBuffer()
assert_true(result instanceof ArrayBuffer, 'Result should be an ArrayBuffer')
assert_array_equals(new Uint8Array(result), expected)
t.done()
})
}
// Assert that two TypedArray objects have the same byte values
globalThis.assert_equals_typed_array = (array1, array2) => {
const [view1, view2] = [array1, array2].map((array) => {
assert_true(array.buffer instanceof ArrayBuffer,
'Expect input ArrayBuffers to contain field `buffer`');
return new DataView(array.buffer, array.byteOffset, array.byteLength);
});
'Expect input ArrayBuffers to contain field `buffer`')
return new DataView(array.buffer, array.byteOffset, array.byteLength)
})
assert_equals(view1.byteLength, view2.byteLength,
'Expect both arrays to be of the same byte length');
'Expect both arrays to be of the same byte length')
const byteLength = view1.byteLength;
const byteLength = view1.byteLength
for (let i = 0; i < byteLength; ++i) {
assert_equals(view1.getUint8(i), view2.getUint8(i),
`Expect byte at buffer position ${i} to be equal`);
`Expect byte at buffer position ${i} to be equal`)
}
}
let hasFailed
globalThis.add_result_callback((test, ...args) => {
if ([
'Blob with type "A"',
'Blob with type "TEXT/HTML"',
'Getters and value conversions should happen in order until an exception is thrown.',
'Using type in File constructor: TEXT/PLAIN',
'Using type in File constructor: text/plain;charset=UTF-8'
].includes(test.name)) return
globalThis.add_result_callback(test => {
const INDENT_SIZE = 2;
var reporter = {}
if (test.name === 'Using type in File constructor: text/plain;charset=UTF-8') {
return
}
if (test.name === 'Using type in File constructor: TEXT/PLAIN') {
return
}
const INDENT_SIZE = 2
const reporter = {}
reporter.startSuite = name => console.log(`\n ${(name)}\n`);
reporter.startSuite = name => console.log(`\n ${(name)}\n`)
reporter.pass = message => console.log((indent(("√ ") + message, INDENT_SIZE)));
reporter.pass = message => console.log((indent(('√ ') + message, INDENT_SIZE)))
reporter.fail = message => console.log((indent("\u00D7 " + message, INDENT_SIZE)));
reporter.fail = message => console.log((indent('\u00D7 ' + message, INDENT_SIZE)))
reporter.reportStack = stack => console.log((indent(stack, INDENT_SIZE * 2)));
reporter.reportStack = stack => console.log((indent(stack, INDENT_SIZE * 2)))
function indent(string, times) {
const prefix = " ".repeat(times);
return string.split("\n").map(l => prefix + l).join("\n");
function indent (string, times) {
const prefix = ' '.repeat(times)
return string.split('\n').map(l => prefix + l).join('\n')
}
if (test.status === 0) {
reporter.pass(test.name);
reporter.pass(test.name)
} else if (test.status === 1) {
reporter.fail(`${test.name}\n`);
reporter.reportStack(`${test.message}\n${test.stack}`);
hasFailed = true;
reporter.fail(`${test.name}\n`)
reporter.reportStack(`${test.message}\n${test.stack}`)
hasFailed = true
} else if (test.status === 2) {
reporter.fail(`${test.name} (timeout)\n`);
reporter.reportStack(`${test.message}\n${test.stack}`);
hasFailed = true;
reporter.fail(`${test.name} (timeout)\n`)
reporter.reportStack(`${test.message}\n${test.stack}`)
hasFailed = true
} else if (test.status === 3) {
reporter.fail(`${test.name} (incomplete)\n`);
reporter.reportStack(`${test.message}\n${test.stack}`);
hasFailed = true;
reporter.fail(`${test.name} (incomplete)\n`)
reporter.reportStack(`${test.message}\n${test.stack}`)
hasFailed = true
} else if (test.status === 4) {
reporter.fail(`${test.name} (precondition failed)\n`);
reporter.reportStack(`${test.message}\n${test.stack}`);
hasFailed = true;
reporter.fail(`${test.name} (precondition failed)\n`)
reporter.reportStack(`${test.message}\n${test.stack}`)
hasFailed = true
} else {
reporter.fail(`unknown test status: ${test.status}`);
hasFailed = true;
reporter.fail(`unknown test status: ${test.status}`)
hasFailed = true
}
hasFailed && process.exit(1);
})
globalThis.File = File
globalThis.Blob = Blob
globalThis.garbageCollect = () => {}
globalThis.document = {body: '[object HTMLBodyElement]'}
globalThis.test_blob = test_blob;
globalThis.test_blob_binary = test_blob_binary;
globalThis.test_blob = test_blob
globalThis.test_blob_binary = test_blob_binary
// Cuz WPT don't clean up after itself
globalThis.MessageChannel = class extends MessageChannel {
constructor () {
super()
setTimeout(() => {
this.port1.close()
this.port2.close()
this.port1.onmessage = this.port2.onmessage = null
}, 100)
}
}
import("https://wpt.live/FileAPI/file/File-constructor.any.js")
import("https://wpt.live/FileAPI/blob/Blob-array-buffer.any.js")
import("https://wpt.live/FileAPI/blob/Blob-slice-overflow.any.js")
import("https://wpt.live/FileAPI/blob/Blob-slice.any.js")
import("https://wpt.live/FileAPI/blob/Blob-stream.any.js")
import("https://wpt.live/FileAPI/blob/Blob-text.any.js")
import('https://wpt.live/FileAPI/file/File-constructor.any.js')
import('https://wpt.live/FileAPI/blob/Blob-constructor.any.js')
import('https://wpt.live/FileAPI/blob/Blob-array-buffer.any.js')
import('https://wpt.live/FileAPI/blob/Blob-slice-overflow.any.js')
import('https://wpt.live/FileAPI/blob/Blob-slice.any.js')
import('https://wpt.live/FileAPI/blob/Blob-stream.any.js')
import('https://wpt.live/FileAPI/blob/Blob-text.any.js')
import('./own-misc-test.js')
hasFailed && process.exit(1)