Compare commits

...
Author SHA1 Message Date
Jimmy Wärting 46f06235d2 using v4 2021-09-07 16:51:17 +02:00
Jimmy Wärting 46476faf49 Update CHANGELOG.md 2021-07-31 11:30:20 +02:00
Jimmy Wärting b86b1409de found/fixed some errors while testing agains WPT (#109)
* found/fixed some errors while testing agains WPT
* only run test.js with ava
2021-07-31 11:24:15 +02:00
Jimmy Wärting a08f394f1b Create PULL_REQUEST_TEMPLATE.md 2021-07-31 11:08:57 +02:00
Nick K c2903ca530 Allow usage of iterable object in Blob constructor. (#108)
* Allow usage of iterable object in Blob constructor.

* Fix ESLint errors
2021-07-29 15:42:42 +02:00
Jimmy Wärting 0b028432d0 bumped patch 2021-07-16 11:59:39 +02:00
Jimmy Wärting f64c20694f added streams.cjs 2021-07-16 11:58:08 +02:00
Jimmy Wärting 613160ba58 bumped patch 2021-07-16 11:57:08 +02:00
Jimmy Wärting efdad69698 bumped patch 2021-07-16 11:55:56 +02:00
Jimmy Wärting e00f849f6d improve typing (#105)
* improve typing

* bug fix

* don't test cjs
2021-07-16 11:51:34 +02:00
Jimmy Wärting 52f296c283 Conditional loading (#104)
use good old cjs to conditionally load stuff in node only
2021-07-16 00:33:00 +02:00
Jimmy Wärting 701103ffd2 Update CHANGELOG.md 2021-07-15 22:55:57 +02:00
Jimmy Wärting ada23da6ee Update package.json 2021-07-15 22:50:37 +02:00
Jimmy Wärting 16f312c543 use whatwg streams (#103)
* use whatwg streams
* could not use optional changing in v12
* could not use fs/promise in v12
2021-07-15 22:49:58 +02:00
Jimmy Wärting 5ba554efd2 Lowered targeted node version to those who support ESM
closes #98
2021-06-19 11:51:54 +02:00
Jimmy Wärting 4747497b52 fixed lastModified 2021-06-10 10:51:16 +02:00
Jimmy Wärting 09fd40448a Update file.js 2021-06-10 10:33:00 +02:00
Jimmy Wärting 8d14e88040 Created a File Class (#97)
* Created a File Class
* Test the untested parts
* ignore any d.ts file
2021-05-29 11:17:26 +02:00
dependabot-preview[bot]anddependabot-preview[bot] <27856297+dependabot-preview[bot]@users.noreply.github.com> 8ef89adad4 Upgrade to GitHub-native Dependabot (#89)
Co-authored-by: dependabot-preview[bot] <27856297+dependabot-preview[bot]@users.noreply.github.com>
2021-05-14 11:01:14 +02:00
13 changed files with 656 additions and 134 deletions
+24
View File
@@ -0,0 +1,24 @@
<!-- Thanks for contributing! -->
## The purpose of this PR is:
...
## This is what had to change:
...
## This is what like reviewers to know:
...
-------------------------------------------------------------------------------------------------
<!-- Mark what you have done, Remove unnecessary ones. Add new tasks that may fit (like TODO's) -->
- [ ] I prefixed the PR-title with `docs: `, `fix(area): `, `feat(area): ` or `breaking(area): `
- [ ] I updated ./CHANGELOG.md with a link to this PR or Issue
- [ ] I updated the README.md
- [ ] I Added unit test(s)
-------------------------------------------------------------------------------------------------
<!-- Add a `- fix #_NUMBER_` line for every Issue this PR solves. Do not comma separate them -->
- fix #000
+25
View File
@@ -0,0 +1,25 @@
version: 2
updates:
- package-ecosystem: npm
directory: "/"
schedule:
interval: daily
open-pull-requests-limit: 10
ignore:
- dependency-name: get-stream
versions:
- 6.0.0
- 6.0.1
- dependency-name: typescript
versions:
- 4.1.3
- 4.1.4
- 4.1.5
- 4.2.2
- 4.2.3
- 4.2.4
- dependency-name: xo
versions:
- 0.37.1
- 0.38.1
- 0.38.2
+1 -2
View File
@@ -62,5 +62,4 @@ typings/
# dotenv environment variables file
.env
index.d.ts
from.d.ts
*.d.ts
+38 -6
View File
@@ -1,18 +1,47 @@
Changelog
=========
# Changelog
## v3.0.0-rc.0
All notable changes to this project will be documented in this file.
The format is based on [Keep a Changelog](http://keepachangelog.com/) and this
project adheres to [Semantic Versioning](http://semver.org/).
(Unreleased)
==================
### Removed
-
### Changed
-
### Added
-
### Fixed
- Allow usage of iterable object in Blob constructor. [#108]
- Run test WPT test against our impl [#109]
- File name are now casted to string [#109]
- Slicing in the middle of multiple parts added more bytes than what what it should have [#109]
## v3.1.2
- Improved typing
- Fixed a bug where position in iterator did not increase
## v3.1.0
- started to use real whatwg streams
- degraded fs/promise to fs.promise to support node v12
- degraded optional changing to support node v12
## v3.0.0
- Changed WeakMap for private field (require node 12)
- Switch to ESM
- blob.stream() return a subset of whatwg stream which is the async iterable
- blob.stream() return a subset of whatwg stream which is the async iterable part
(it no longer return a node stream)
- Reduced the dependency of Buffer by changing to global TextEncoder/Decoder (require node 11)
- Disabled xo since it could understand private fields (#)
- No longer transform the type to lowercase (https://github.com/w3c/FileAPI/issues/43)
This is more loose than strict, keys should be lowercased, but values should not.
It would require a more proper mime type parser - so we just made it loose.
- index.js can now be imported by browser & deno since it no longer depends on any
core node features (but why would you? other environment can benefit from it)
- index.js and file.js can now be imported by browser & deno since it no longer depends on any
core node features (but why would you?)
- Implemented a File class
## v2.1.2
- Fixed a bug where `start` in BlobDataItem was undefined (#85)
@@ -69,3 +98,6 @@ Changelog
## v1.0.0
- Major: initial release
[#108]: https://github.com/node-fetch/fetch-blob/pull/108
[#109]: https://github.com/node-fetch/fetch-blob/pull/109
+27 -59
View File
@@ -22,10 +22,8 @@ npm install fetch-blob
- internal Buffer.from was replaced with TextEncoder/Decoder
- internal buffers was replaced with Uint8Arrays
- CommonJS was replaced with ESM
- The node stream returned by calling `blob.stream()` was replaced with a simple generator function that yields Uint8Array (Breaking change)
(Read "Differences from other blobs" for more info.)
All of this changes have made it dependency free of any core node modules, so it would be possible to just import it using http-import from a CDN without any bundling
- The node stream returned by calling `blob.stream()` was replaced with whatwg streams
- (Read "Differences from other blobs" for more info.)
</details>
@@ -36,48 +34,12 @@ npm install fetch-blob
- This blob version is more arbitrary, it can be constructed with blob parts that isn't a instance of itself
it has to look and behave as a blob to be accepted as a blob part.
- The benefit of this is that you can create other types of blobs that don't contain any internal data that has to be read in other ways, such as the `BlobDataItem` created in `from.js` that wraps a file path into a blob-like item and read lazily (nodejs plans to [implement this][fs-blobs] as well)
- The `blob.stream()` is the most noticeable differences. It returns a AsyncGeneratorFunction that yields Uint8Arrays
- The `blob.stream()` is the most noticeable differences. It returns a WHATWG stream now. to keep it as a node stream you would have to do:
The reasoning behind `Blob.prototype.stream()` is that NodeJS readable stream
isn't spec compatible with whatwg streams and we didn't want to import the hole whatwg stream polyfill for node
or browserify NodeJS streams for the browsers and picking any flavor over the other. So we decided to opted out
of any stream and just implement the bear minium of what both streams have in common which is the asyncIterator
that both yields Uint8Array. this is the most isomorphic way with the use of `for-await-of` loops.
It would be redundant to convert anything to whatwg streams and than convert it back to
node streams since you work inside of Node.
It will probably stay like this until nodejs get native support for whatwg<sup>[1][https://github.com/nodejs/whatwg-stream]</sup> streams and whatwg stream add the node
equivalent for `Readable.from(iterable)`<sup>[2](https://github.com/whatwg/streams/issues/1018)</sup>
But for now if you really need a Node Stream then you can do so using this transformation
```js
import {Readable} from 'stream'
const stream = Readable.from(blob.stream())
```
But if you don't need it to be a stream then you can just use the asyncIterator part of it that is isomorphic.
```js
for await (const chunk of blob.stream()) {
console.log(chunk) // uInt8Array
}
```
If you need to make some feature detection to fix this different behavior
```js
if (Blob.prototype.stream?.constructor?.name === 'AsyncGeneratorFunction') {
// not spec compatible, monkey patch it...
// (Alternative you could extend the Blob and use super.stream())
let orig = Blob.prototype.stream
Blob.prototype.stream = function () {
const iterator = orig.call(this)
return new ReadableStream({
async pull (ctrl) {
const next = await iterator.next()
return next.done ? ctrl.close() : ctrl.enqueue(next.value)
}
})
}
}
```
Possible feature whatwg version: `ReadableStream.from(iterator)`
It's also possible to delete this method and instead use `.slice()` and `.arrayBuffer()` since it has both a public and private stream method
</details>
## Usage
@@ -86,7 +48,11 @@ npm install fetch-blob
// Ways to import
// (PS it's dependency free ESM package so regular http-import from CDN works too)
import Blob from 'fetch-blob'
import File from 'fetch-blob/file.js'
import {Blob} from 'fetch-blob'
import {File} from 'fetch-blob/file.js'
const {Blob} = await import('fetch-blob')
@@ -96,36 +62,38 @@ const blob = new Blob(['hello, world'])
await blob.text()
await blob.arrayBuffer()
for await (let chunk of blob.stream()) { ... }
// turn the async iterator into a node stream
stream.Readable.from(blob.stream())
// turn the async iterator into a whatwg stream (feature)
globalThis.ReadableStream.from(blob.stream())
blob.stream().getReader().read()
blob.stream().getReader({mode: 'byob'}).read(view)
```
### Blob part backed up by filesystem
To use, install [domexception](https://github.com/jsdom/domexception).
```sh
npm install fetch-blob domexception
```
`fetch-blob/from.js` comes packed with tools to convert any filepath into either a Blob or a File
It will not read the content into memory. It will only stat the file for last modified date and file size.
```js
// The default export is sync and use fs.stat to retrieve size & last modified
// The default export is sync and use fs.stat to retrieve size & last modified as a blob
import blobFromSync from 'fetch-blob/from.js'
import {Blob, blobFrom, blobFromSync} from 'fetch-blob/from.js'
import {File, Blob, blobFrom, blobFromSync, fileFrom, fileFromSync} from 'fetch-blob/from.js'
const fsBlob1 = blobFromSync('./2-GiB-file.bin')
const fsBlob2 = await blobFrom('./2-GiB-file.bin')
const fsFile = fileFromSync('./2-GiB-file.bin', 'application/octet-stream')
const fsBlob = await blobFrom('./2-GiB-file.mp4')
// Not a 4 GiB memory snapshot, just holds 3 references
// Not a 4 GiB memory snapshot, just holds references
// points to where data is located on the disk
const blob = new Blob([fsBlob1, fsBlob2, 'memory'])
console.log(blob.size) // 4 GiB
const blob = new Blob([fsFile, fsBlob, 'memory', new Uint8Array(10)])
console.log(blob.size) // ~4 GiB
```
See the [MDN documentation](https://developer.mozilla.org/en-US/docs/Web/API/Blob) and [tests](https://github.com/node-fetch/fetch-blob/blob/master/test.js) for more details.
`blobFrom|blobFromSync|fileFrom|fileFromSync(path, [mimetype])`
### Creating Blobs backed up by other async sources
Our Blob & File class are more generic then any other polyfills in the way that it can accept any blob look-a-like item
An example of this is that our blob implementation can be constructed with parts coming from [BlobDataItem](https://github.com/node-fetch/fetch-blob/blob/8ef89adad40d255a3bbd55cf38b88597c1cd5480/from.js#L32) (aka a filepath) or from [buffer.Blob](https://nodejs.org/api/buffer.html#buffer_new_buffer_blob_sources_options), It dose not have to implement all the methods - just enough that it can be read/understood by our Blob implementation. The minium requirements is that it has `Symbol.toStringTag`, `size`, `slice()` and either a `stream()` or a `arrayBuffer()` method. If you then wrap it in our Blob or File `new Blob([blobDataItem])` then you get all of the other methods that should be implemented in a blob or file
An example of this could be to create a file or blob like item coming from a remote HTTP request. Or from a DataBase
See the [MDN documentation](https://developer.mozilla.org/en-US/docs/Web/API/Blob) and [tests](https://github.com/node-fetch/fetch-blob/blob/master/test.js) for more details of how to use the Blob.
[npm-image]: https://flat.badgen.net/npm/v/fetch-blob
[npm-url]: https://www.npmjs.com/package/fetch-blob
+40
View File
@@ -0,0 +1,40 @@
import Blob from './index.js';
const _File = class File extends Blob {
#lastModified = 0;
#name = '';
/**
* @param {*[]} fileBits
* @param {string} fileName
* @param {{lastModified?: number, type?: string}} options
*/// @ts-ignore
constructor(fileBits, fileName, options = {}) {
if (arguments.length < 2) {
throw new TypeError(`Failed to construct 'File': 2 arguments required, but only ${arguments.length} present.`);
}
super(fileBits, options);
if (options === null) options = {};
const modified = Number(options.lastModified);
this.#lastModified = Number.isNaN(modified) ? Date.now() : modified
this.#name = String(fileName);
}
get name() {
return this.#name;
}
get lastModified() {
return this.#lastModified;
}
get [Symbol.toStringTag]() {
return "File";
}
}
/** @type {typeof globalThis.File} */// @ts-ignore
export const File = _File;
export default File;
+49 -19
View File
@@ -1,26 +1,58 @@
import {statSync, createReadStream} from 'fs';
import {stat} from 'fs/promises';
import DOMException from 'domexception';
import {statSync, createReadStream, promises as fs} from 'fs';
import {basename} from 'path';
import {MessageChannel} from 'worker_threads';
import File from './file.js';
import Blob from './index.js';
/**
* @param {string} path filepath on the disk
* @returns {Blob}
*/
const blobFromSync = path => from(statSync(path), path);
const {stat} = fs;
const DOMException = globalThis.DOMException || (() => {
const port = new MessageChannel().port1
const ab = new ArrayBuffer(0)
try { port.postMessage(ab, [ab, ab]) }
catch (err) { return err.constructor }
})()
/**
* @param {string} path filepath on the disk
* @returns {Promise<Blob>}
* @param {string} [type] mimetype to use
*/
const blobFrom = path => stat(path).then(stat => from(stat, path));
const blobFromSync = (path, type) => fromBlob(statSync(path), path, type);
const from = (stat, path) => new Blob([new BlobDataItem({
/**
* @param {string} path filepath on the disk
* @param {string} [type] mimetype to use
*/
const blobFrom = (path, type) => stat(path).then(stat => fromBlob(stat, path, type));
/**
* @param {string} path filepath on the disk
* @param {string} [type] mimetype to use
*/
const fileFrom = (path, type) => stat(path).then(stat => fromFile(stat, path, type));
/**
* @param {string} path filepath on the disk
* @param {string} [type] mimetype to use
*/
const fileFromSync = (path, type) => fromFile(statSync(path), path, type);
// @ts-ignore
const fromBlob = (stat, path, type = '') => new Blob([new BlobDataItem({
path,
size: stat.size,
lastModified: stat.mtimeMs,
start: 0
})]);
})], {type});
// @ts-ignore
const fromFile = (stat, path, type = '') => new File([new BlobDataItem({
path,
size: stat.size,
lastModified: stat.mtimeMs,
start: 0
})], basename(path), { type, lastModified: stat.mtimeMs });
/**
* This is a blob backed up by a file on the disk
@@ -58,12 +90,10 @@ class BlobDataItem {
if (mtimeMs > this.lastModified) {
throw new DOMException('The requested file could not be read, typically due to permission problems that have occurred after a reference to a file was acquired.', 'NotReadableError');
}
if (this.size) {
yield * createReadStream(this.#path, {
start: this.#start,
end: this.#start + this.size - 1
});
}
yield * createReadStream(this.#path, {
start: this.#start,
end: this.#start + this.size - 1
});
}
get [Symbol.toStringTag]() {
@@ -72,4 +102,4 @@ class BlobDataItem {
}
export default blobFromSync;
export {Blob, blobFrom, blobFromSync};
export {File, Blob, blobFrom, blobFromSync, fileFrom, fileFromSync};
+54 -24
View File
@@ -1,7 +1,15 @@
// TODO (jimmywarting): in the feature use conditional loading with top level await (requires 14.x)
// Node has recently added whatwg stream into core
import './streams.cjs';
/** @typedef {import('buffer').Blob} NodeBlob} */
// 64 KiB (same size chrome slice theirs blob into Uint8array's)
const POOL_SIZE = 65536;
/** @param {(Blob | Uint8Array)[]} parts */
/** @param {(Blob | NodeBlob | Uint8Array)[]} parts */
async function * toIterator (parts, clone = true) {
for (let part of parts) {
if ('stream' in part) {
@@ -13,13 +21,14 @@ async function * toIterator (parts, clone = true) {
while (position !== end) {
const size = Math.min(end - position, POOL_SIZE);
const chunk = part.buffer.slice(position, position + size);
yield new Uint8Array(chunk);
position += chunk.byteLength;
yield new Uint8Array(chunk);
}
} else {
yield part;
}
} else {
/* c8 ignore start */
// For blobs that have arrayBuffer but no stream method (nodes buffer.Blob)
let position = 0;
while (position !== part.size) {
@@ -28,11 +37,12 @@ async function * toIterator (parts, clone = true) {
position += buffer.byteLength;
yield new Uint8Array(buffer);
}
/* c8 ignore end */
}
}
}
export default class Blob {
const _Blob = class Blob {
/** @type {Array.<(Blob|Uint8Array)>} */
#parts = [];
@@ -48,9 +58,19 @@ export default class Blob {
* @param {{ type?: string }} [options]
*/
constructor(blobParts = [], options = {}) {
const parts = [];
let size = 0;
if (typeof blobParts !== 'object') {
throw new TypeError(`Failed to construct 'Blob': parameter 1 is not an iterable object.`);
}
const parts = blobParts.map(element => {
if (typeof options !== 'object' && typeof options !== 'function') {
throw new TypeError(`Failed to construct 'Blob': parameter 2 cannot convert to dictionary.`);
}
if (options === null) options = {};
for (const element of blobParts) {
let part;
if (ArrayBuffer.isView(element)) {
part = new Uint8Array(element.buffer.slice(element.byteOffset, element.byteOffset + element.byteLength));
@@ -59,16 +79,16 @@ export default class Blob {
} else if (element instanceof Blob) {
part = element;
} else {
part = new TextEncoder().encode(String(element));
part = new TextEncoder().encode(element);
}
size += ArrayBuffer.isView(part) ? part.byteLength : part.size;
return part;
});
parts.push(part);
}
const type = options.type === undefined ? '' : String(options.type);
this.#type = /[^\u0020-\u007E]/.test(type) ? '' : type;
this.#type = /^[\x20-\x7E]*$/.test(type) ? type : '';
this.#size = size;
this.#parts = parts;
}
@@ -116,6 +136,11 @@ export default class Blob {
* @return {Promise<ArrayBuffer>}
*/
async arrayBuffer() {
// Easier way... Just a unnecessary overhead
// const view = new Uint8Array(this.size);
// await this.stream().getReader({mode: 'byob'}).read(view);
// return view.buffer;
const data = new Uint8Array(this.size);
let offset = 0;
for await (const chunk of toIterator(this.#parts, false)) {
@@ -126,14 +151,16 @@ export default class Blob {
return data.buffer;
}
/**
* The Blob stream() implements partial support of the whatwg stream
* by only being async iterable.
*
* @returns {AsyncGenerator<Uint8Array>}
*/
async * stream() {
yield * toIterator(this.#parts, true);
stream() {
const it = toIterator(this.#parts, true);
return new ReadableStream({
type: 'bytes',
async pull(ctrl) {
const chunk = await it.next();
chunk.done ? ctrl.close() : ctrl.enqueue(chunk.value);
}
})
}
/**
@@ -157,6 +184,11 @@ export default class Blob {
let added = 0;
for (const part of parts) {
// don't add the overflow to new blobParts
if (added >= span) {
break;
}
const size = ArrayBuffer.isView(part) ? part.byteLength : part.size;
if (relativeStart && size <= relativeStart) {
// Skip the beginning and change the relative
@@ -172,13 +204,9 @@ export default class Blob {
chunk = part.slice(relativeStart, Math.min(size, relativeEnd));
added += chunk.size
}
relativeEnd -= size;
blobParts.push(chunk);
relativeStart = 0; // All next sequental parts should start at 0
// don't add the overflow to new blobParts
if (added >= span) {
break;
}
relativeStart = 0; // All next sequential parts should start at 0
}
}
@@ -207,10 +235,12 @@ export default class Blob {
}
}
Object.defineProperties(Blob.prototype, {
Object.defineProperties(_Blob.prototype, {
size: {enumerable: true},
type: {enumerable: true},
slice: {enumerable: true}
});
export { Blob };
/** @type {typeof globalThis.Blob} */
export const Blob = _Blob;
export default Blob;
+34 -19
View File
@@ -1,31 +1,36 @@
{
"name": "fetch-blob",
"version": "3.0.0-rc.0",
"description": "A Blob implementation in Node.js, originally from node-fetch.",
"version": "3.1.2",
"description": "Blob & File implementation in Node.js, originally from node-fetch.",
"main": "index.js",
"type": "module",
"files": [
"from.js",
"file.js",
"file.d.ts",
"index.js",
"index.d.ts",
"from.d.ts"
"from.d.ts",
"streams.cjs"
],
"scripts": {
"lint": "xo test.js",
"test": "npm run lint && ava",
"report": "c8 --reporter json --reporter text ava",
"coverage": "c8 --reporter json --reporter text ava && codecov -f coverage/coverage-final.json",
"test-wpt": "node --experimental-loader ./test/http-loader.js ./test/test-wpt-in-node.js",
"test": "npm run lint && ava test.js",
"report": "c8 --reporter json --reporter text ava test.js",
"coverage": "c8 --reporter json --reporter text ava test.js && codecov -f coverage/coverage-final.json",
"prepublishOnly": "tsc --declaration --emitDeclarationOnly --allowJs index.js from.js"
},
"repository": "https://github.com/node-fetch/fetch-blob.git",
"keywords": [
"blob",
"file",
"node-fetch"
],
"engines": {
"node": ">=14.0.0"
"node": "^12.20 || >= 14.13"
},
"author": "David Frank",
"author": "Jimmy Wärting <jimmy@warting.se> (https://jimmy.warting.se)",
"license": "MIT",
"bugs": {
"url": "https://github.com/node-fetch/fetch-blob/issues"
@@ -33,7 +38,10 @@
"homepage": "https://github.com/node-fetch/fetch-blob#readme",
"xo": {
"rules": {
"unicorn/import-index": "off",
"unicorn/prefer-node-protocol": "off",
"unicorn/numeric-separators-style": "off",
"unicorn/prefer-spread": "off",
"unicorn/prefer-number-properties": "off",
"import/extensions": [
"error",
"always",
@@ -52,18 +60,25 @@
}
]
},
"peerDependenciesMeta": {
"domexception": {
"optional": true
}
},
"devDependencies": {
"ava": "^3.15.0",
"c8": "^7.7.1",
"codecov": "^3.8.1",
"domexception": "^2.0.1",
"c8": "^7.7.2",
"codecov": "^3.8.2",
"node-fetch": "^3.0.0-beta.9",
"typescript": "^4.2.4",
"xo": "^0.38.2"
"typescript": "^4.3.2",
"xo": "^0.40.1"
},
"funding": [
{
"type": "github",
"url": "https://github.com/sponsors/jimmywarting"
},
{
"type": "paypal",
"url": "https://paypal.me/jimmywarting"
}
],
"dependencies": {
"web-streams-polyfill": "^4.0.0-beta.1"
}
}
+37
View File
@@ -0,0 +1,37 @@
/* c8 ignore start */
// 64 KiB (same size chrome slice theirs blob into Uint8array's)
const POOL_SIZE = 65536;
if (!globalThis.ReadableStream) {
try {
Object.assign(globalThis, require('stream/web'))
} catch (error) {
// TODO: Remove when only supporting node >= 16.5.0
require('web-streams-polyfill/polyfill')
}
}
try {
const {Blob} = require('buffer')
if (Blob && !Blob.prototype.stream) {
Blob.prototype.stream = function name(params) {
let position = 0;
const blob = this;
return new ReadableStream({
type: 'bytes',
async pull(ctrl) {
const chunk = blob.slice(position, Math.min(blob.size, position + POOL_SIZE));
const buffer = await chunk.arrayBuffer();
position += buffer.byteLength;
ctrl.enqueue(new Uint8Array(buffer))
if (position === blob.size) {
ctrl.close()
}
}
})
}
}
} catch (error) {}
/* c8 ignore end */
+143 -5
View File
@@ -1,10 +1,11 @@
import fs from 'fs';
import test from 'ava';
import {Response} from 'node-fetch';
import {Readable} from 'stream';
import buffer from 'buffer';
import test from 'ava';
import {Response} from 'node-fetch';
import syncBlob, {blobFromSync, blobFrom, fileFromSync, fileFrom} from './from.js';
import File from './file.js';
import Blob from './index.js';
import syncBlob, {blobFromSync, blobFrom} from './from.js';
const license = fs.readFileSync('./LICENSE', 'utf-8');
@@ -35,6 +36,38 @@ test('Blob ctor parts', async t => {
t.is(await blob.text(), 'abcdefg[object Object]foo=');
});
test('Blob ctor threats an object with @@iterator as a sequence', async t => {
const blob = new Blob({[Symbol.iterator]: Array.prototype[Symbol.iterator]});
t.is(blob.size, 0);
t.is(await blob.text(), '');
});
test('Blob ctor reads blob parts from object with @@iterator', async t => {
const input = ['one', 'two', 'three'];
const expected = input.join('');
const blob = new Blob({
* [Symbol.iterator]() {
yield * input;
}
});
t.is(blob.size, new TextEncoder().encode(expected).byteLength);
t.is(await blob.text(), expected);
});
test('Blob ctor throws a string', t => {
t.throws(() => new Blob('abc'));
});
test('Blob ctor threats Uint8Array as a sequence', async t => {
const input = [1, 2, 3];
const blob = new Blob(new Uint8Array(input));
t.is(await blob.text(), input.join(''));
});
test('Blob size', t => {
const data = 'a=1';
const blob = new Blob([data]);
@@ -158,13 +191,34 @@ test('blob part backed up by filesystem', async t => {
test('Reading after modified should fail', async t => {
const blob = blobFromSync('./LICENSE');
await new Promise(resolve => {
setTimeout(resolve, 500);
});
fs.closeSync(fs.openSync('./LICENSE', 'a'));
const error = await t.throwsAsync(blob.text());
t.is(error.constructor.name, 'DOMException');
t.is(error instanceof Error, true);
t.is(error.name, 'NotReadableError');
const file = fileFromSync('./LICENSE');
// Above test updates the last modified date to now
t.is(typeof file.lastModified, 'number');
// The lastModifiedDate is deprecated and removed from spec
t.false('lastModifiedDate' in file);
const mod = file.lastModified - Date.now();
t.true(mod <= 0 && mod >= -500); // Close to tolerance: 0.500ms
});
test('Reading file after modified should fail', async t => {
const file = fileFromSync('./LICENSE');
await new Promise(resolve => {
setTimeout(resolve, 100);
});
const now = new Date();
// Change modified time
fs.utimesSync('./LICENSE', now, now);
const error = await blob.text().catch(error => error);
const error = await t.throwsAsync(file.text());
t.is(error.constructor.name, 'DOMException');
t.is(error instanceof Error, true);
t.is(error.name, 'NotReadableError');
});
@@ -219,7 +273,7 @@ test('Parts are immutable', async t => {
test('Blobs are immutable', async t => {
const buf = new Uint8Array([97]);
const blob = new Blob([buf]);
const chunk = await blob.stream().next();
const chunk = await blob.stream().getReader().read();
t.is(chunk.value[0], 97);
chunk.value[0] = 98;
t.is(await blob.text(), 'a');
@@ -239,6 +293,7 @@ test('Large chunks are divided into smaller chunks', async t => {
});
test('Can use named import - as well as default', async t => {
// eslint-disable-next-line node/no-unsupported-features/es-syntax
const {Blob, default: def} = await import('./index.js');
t.is(Blob, def);
});
@@ -254,3 +309,86 @@ if (buffer.Blob) {
t.is(await blob2.text(), 'blob part');
});
}
test('File is a instance of blob', t => {
t.true(new File([], '') instanceof Blob);
});
test('fileFrom returns the name', async t => {
t.is((await fileFrom('./LICENSE')).name, 'LICENSE');
});
test('fileFromSync returns the name', t => {
t.is(fileFromSync('./LICENSE').name, 'LICENSE');
});
test('fileFromSync(path, type) sets the type', t => {
t.is(fileFromSync('./LICENSE', 'text/plain').type, 'text/plain');
});
test('blobFromSync(path, type) sets the type', t => {
t.is(blobFromSync('./LICENSE', 'text/plain').type, 'text/plain');
});
test('fileFrom(path, type) sets the type', async t => {
const file = await fileFrom('./LICENSE', 'text/plain');
t.is(file.type, 'text/plain');
});
test('new File(,,{lastModified: 100})', t => {
const mod = new File([], '', {lastModified: 100}).lastModified;
t.is(mod, 100);
});
test('new File(,,{lastModified: "200"})', t => {
const mod = new File([], '', {lastModified: '200'}).lastModified;
t.is(mod, 200);
});
test('new File(,,{lastModified: true})', t => {
const mod = new File([], '', {lastModified: true}).lastModified;
t.is(mod, 1);
});
test('new File(,,{lastModified: new Date()})', t => {
const mod = new File([], '', {lastModified: new Date()}).lastModified - Date.now();
t.true(mod <= 0 && mod >= -20); // Close to tolerance: 0.020ms
});
test('new File(,,{}) sets current time', t => {
const mod = new File([], '').lastModified - Date.now();
t.true(mod <= 0 && mod >= -20); // Close to tolerance: 0.020ms
});
test('blobFrom(path, type) sets the type', async t => {
const blob = await blobFrom('./LICENSE', 'text/plain');
t.is(blob.type, 'text/plain');
});
test('blobFrom(path) sets empty type', async t => {
const blob = await blobFrom('./LICENSE');
t.is(blob.type, '');
});
test('new File() throws with too few args', t => {
t.throws(() => new File(), {
instanceOf: TypeError,
message: 'Failed to construct \'File\': 2 arguments required, but only 0 present.'
});
});
test('can slice zero sized blobs', async t => {
const blob = new Blob();
const txt = await blob.slice(0, 0).text();
t.is(txt, '');
});
test('returns a readable stream', t => {
const stream = new File([], '').stream();
t.true(typeof stream.getReader === 'function');
});
test('checking instanceof blob#stream', t => {
const stream = new File([], '').stream();
t.true(stream instanceof globalThis.ReadableStream);
});
+51
View File
@@ -0,0 +1,51 @@
// https-loader.mjs
import { get } from 'https';
export function resolve(specifier, context, defaultResolve) {
const { parentURL = null } = context;
// Normally Node.js would error on specifiers starting with 'https://', so
// this hook intercepts them and converts them into absolute URLs to be
// passed along to the later hooks below.
if (specifier.startsWith('https://')) {
return {
url: specifier
};
} else if (parentURL && parentURL.startsWith('https://')) {
return {
url: new URL(specifier, parentURL).href
};
}
// Let Node.js handle all other specifiers.
return defaultResolve(specifier, context, defaultResolve);
}
export function getFormat(url, context, defaultGetFormat) {
// This loader assumes all network-provided JavaScript is ES module code.
if (url.startsWith('https://')) {
return {
format: 'module'
};
}
// Let Node.js handle all other URLs.
return defaultGetFormat(url, context, defaultGetFormat);
}
export function getSource(url, context, defaultGetSource) {
// For JavaScript to be loaded over the network, we need to fetch and
// return it.
if (url.startsWith('https://')) {
return new Promise((resolve, reject) => {
let data = ''
get(url, async res => {
for await (const chunk of res) data += chunk;
resolve({ source: data });
}).on('error', (err) => reject(err));
});
}
// Let Node.js handle all other URLs.
return defaultGetSource(url, context, defaultGetSource);
}
+133
View File
@@ -0,0 +1,133 @@
// Don't want to use the FileReader, don't want to lowerCase the type either
// import from 'https://wpt.live/resources/testharnessreport.js'
import {File, Blob} from '../from.js'
globalThis.self = globalThis
await import('https://wpt.live/resources/testharness.js')
// Should probably be fixed... should be able to compare a Blob to a File
delete Blob[Symbol.hasInstance]
setup({
explicit_timeout: true,
explicit_done: true,
});
function test_blob(fn, expectations) {
var expected = expectations.expected,
type = expectations.type,
desc = expectations.desc;
var t = async_test(desc);
t.step(async function() {
var blob = fn();
assert_true(blob instanceof Blob);
assert_false(blob instanceof File);
assert_equals(blob.type.toLowerCase(), type);
assert_equals(blob.size, expected.length);
assert_equals(await blob.text(), expected);
t.done();
});
}
function test_blob_binary(fn, expectations) {
var expected = expectations.expected,
type = expectations.type,
desc = expectations.desc;
var t = async_test(desc);
t.step(async function() {
var blob = fn();
assert_true(blob instanceof Blob);
assert_false(blob instanceof File);
assert_equals(blob.type.toLowerCase(), type);
assert_equals(blob.size, expected.length);
const result = await blob.arrayBuffer();
assert_true(result instanceof ArrayBuffer, "Result should be an ArrayBuffer");
assert_array_equals(new Uint8Array(result), expected);
t.done();
});
}
// Assert that two TypedArray objects have the same byte values
globalThis.assert_equals_typed_array = (array1, array2) => {
const [view1, view2] = [array1, array2].map((array) => {
assert_true(array.buffer instanceof ArrayBuffer,
'Expect input ArrayBuffers to contain field `buffer`');
return new DataView(array.buffer, array.byteOffset, array.byteLength);
});
assert_equals(view1.byteLength, view2.byteLength,
'Expect both arrays to be of the same byte length');
const byteLength = view1.byteLength;
for (let i = 0; i < byteLength; ++i) {
assert_equals(view1.getUint8(i), view2.getUint8(i),
`Expect byte at buffer position ${i} to be equal`);
}
}
let hasFailed
globalThis.add_result_callback(test => {
const INDENT_SIZE = 2;
var reporter = {}
if (test.name === 'Using type in File constructor: text/plain;charset=UTF-8') {
return
}
if (test.name === 'Using type in File constructor: TEXT/PLAIN') {
return
}
reporter.startSuite = name => console.log(`\n ${(name)}\n`);
reporter.pass = message => console.log((indent(("√ ") + message, INDENT_SIZE)));
reporter.fail = message => console.log((indent("\u00D7 " + message, INDENT_SIZE)));
reporter.reportStack = stack => console.log((indent(stack, INDENT_SIZE * 2)));
function indent(string, times) {
const prefix = " ".repeat(times);
return string.split("\n").map(l => prefix + l).join("\n");
}
if (test.status === 0) {
reporter.pass(test.name);
} else if (test.status === 1) {
reporter.fail(`${test.name}\n`);
reporter.reportStack(`${test.message}\n${test.stack}`);
hasFailed = true;
} else if (test.status === 2) {
reporter.fail(`${test.name} (timeout)\n`);
reporter.reportStack(`${test.message}\n${test.stack}`);
hasFailed = true;
} else if (test.status === 3) {
reporter.fail(`${test.name} (incomplete)\n`);
reporter.reportStack(`${test.message}\n${test.stack}`);
hasFailed = true;
} else if (test.status === 4) {
reporter.fail(`${test.name} (precondition failed)\n`);
reporter.reportStack(`${test.message}\n${test.stack}`);
hasFailed = true;
} else {
reporter.fail(`unknown test status: ${test.status}`);
hasFailed = true;
}
hasFailed && process.exit(1);
})
globalThis.File = File
globalThis.Blob = Blob
globalThis.garbageCollect = () => {}
globalThis.document = {body: '[object HTMLBodyElement]'}
globalThis.test_blob = test_blob;
globalThis.test_blob_binary = test_blob_binary;
import("https://wpt.live/FileAPI/file/File-constructor.any.js")
import("https://wpt.live/FileAPI/blob/Blob-array-buffer.any.js")
import("https://wpt.live/FileAPI/blob/Blob-slice-overflow.any.js")
import("https://wpt.live/FileAPI/blob/Blob-slice.any.js")
import("https://wpt.live/FileAPI/blob/Blob-stream.any.js")
import("https://wpt.live/FileAPI/blob/Blob-text.any.js")