Compare commits

..
52 Commits
Author SHA1 Message Date
Jimmy Wärting a3b0d62b9d corrected the release name 2021-05-07 11:45:20 +02:00
Jimmy Wärting 97a084b5e8 Revamp (#83)
* See CHANGELOG for v3.0.0

* removed get-stream

* enabled linter just for test.js & from.js

* Any blob part is acceptable (jsdoc)
since the default fallback is to cast unknown items into strings

* async stat version

* import from fs/promise instead

* Updated the Readme and code examples

* known differences, buffer.Blob support, private stream, more test

* require node 14

* use fetch v3
2021-05-07 11:42:01 +02:00
Jimmy Wärting a8fed4e338 Return empty a empty stream for zero sized blob (#86)
Return empty a empty stream for zero sized blob
2021-04-29 14:01:01 +02:00
Nick K 483973cf25 Fix for the 'start' range of BlobDataItem when reading its stream (#85)
Set default value for 'start' option
2021-04-26 12:06:27 +02:00
Jimmy Wärting cc929f3714 2.1.1 was already publish on npm 9 month ago 2021-04-20 17:52:55 +02:00
Jimmy Wärting 5584886f0f no message 2021-04-20 17:46:23 +02:00
Jimmy Wärting 0d2cd8b9b7 Updated dependencies
Gosh i hate linters. Want to remove them...
2021-04-20 16:32:18 +02:00
Jimmy Wärting 149030ddbe use extension 2021-04-20 16:15:04 +02:00
Nick K f08e6a6773 Add generated typings for from.js file. (#80)
* Generate typings for from.js
* Add generated typings to files section and fix linter issue.
* Remove domexception installation from ci workflow
* Fix for .d.ts output
Co-authored-by: Jimmy Wärting <jimmy@warting.se>
2021-04-20 16:10:46 +02:00
Nick K eed6a1dddd Add nullish values checking in Symbol.hasInstance (#82)
* Add nullish values checking in Symbol.hasInstance
* Fix for a test description
* Add an extension.
* Disable unicorn/import-index rule.
* Fix a typo
2021-04-20 16:07:38 +02:00
Jimmy Wärting 3b12a39ea2 cast type to string 2021-02-13 15:50:02 +01:00
Jimmy Wärting 591a28ed94 Update from.js 2021-02-13 15:01:14 +01:00
Richie Bendall 40f4568c95 Revert 689bee122b 2021-01-29 02:46:12 +13:00
Richie Bendall a74fb90d95 Update ci.yml 2021-01-28 13:57:22 +13:00
Richie Bendall 689bee122b Use native Blob when possible 2021-01-28 13:55:33 +13:00
Richie Bendall 50b89fcdce Make domexception an optional peer dependency 2021-01-18 22:23:51 +13:00
Jimmy Wärting 40c6ff45df Update from.js 2021-01-04 01:00:19 +01:00
Jimmy Wärting 5b2bab5365 Update CHANGELOG.md 2020-07-28 17:48:58 +02:00
Jimmy Wärting b75fb395b4 bumped minor version to 1.2.0 2020-07-28 17:44:52 +02:00
Jimmy Wärting 3217da9bc7 Create blob backed up by filesystem (#55)
* create a blob backed up by filesystem
* added code coverage test for filesystem blobs
* updated readme for using filesystem
* install peer dep
* added from to files array in package.json
2020-07-28 17:42:31 +02:00
Jimmy Wärting ae6b565ae6 fix slice test
`added` bytes was calculated wrong
2020-07-24 01:56:18 +02:00
Irakli Gozalishvili 26ca0e39b4 create test illustrating a bug 2020-07-24 01:56:18 +02:00
Antoni Kepinski 99388a79bb update changelog 2020-06-11 16:31:21 +02:00
Antoni Kepinski cf379d10e0 2.0.1 2020-06-11 16:30:40 +02:00
Pim 1d1faebe18 fix: remove upper bound for node engine semver (#49) 2020-06-11 16:29:53 +02:00
Konstantin Vyatkin 903a4c1bb0 use Symbol.hasInstance (#46) 2020-06-11 10:00:58 -04:00
Konstantin Vyatkin 126b4ec697 Switch to GitHub Actions (#47)
* remove travis

* add codecov partial line coverage

* switch coverage to c8

* add GitHub Actions

* update badge

* import TextDecoder from util

* lint
2020-06-11 10:01:22 +02:00
Antoni Kepinski 38639ec4aa release 2.0.0 2020-06-10 21:09:47 +02:00
Antoni Kepinski 5608cef75c update changelog 2020-06-10 21:08:54 +02:00
Antoni Kepinski 62fc8eab22 1.0.7 2020-06-10 16:14:25 +02:00
Antoni Kepinski 1a3c424a07 update nyc 2020-06-10 16:13:07 +02:00
Antoni Kepinski f4fa7e5825 Update CHANGELOG.md 2020-06-10 16:12:41 +02:00
Konstantin Vyatkin a1fef8cd0b Create TypeScript declarations from JSDoc (#45) 2020-06-10 16:09:02 +02:00
dependabot-preview[bot]anddependabot-preview[bot] <27856297+dependabot-preview[bot]@users.noreply.github.com> 1ec92a63b2 Bump xo from 0.30.0 to 0.32.0 (#39)
Bumps [xo](https://github.com/xojs/xo) from 0.30.0 to 0.32.0.
- [Release notes](https://github.com/xojs/xo/releases)
- [Commits](https://github.com/xojs/xo/compare/v0.30.0...v0.32.0)

Signed-off-by: dependabot-preview[bot] <support@dependabot.com>

Co-authored-by: dependabot-preview[bot] <27856297+dependabot-preview[bot]@users.noreply.github.com>
2020-06-09 20:22:03 +02:00
Jimmy Wärting fd4f45e70f Operate on blob parts (byte sequence) (#44)
* operate on blob parts

* jsdoc

* specify parts type

* Back to 100% (discovered a small bug)
2020-06-07 23:42:42 +02:00
Richie Bendall 9dc07473c6 Update formatting 2020-06-08 01:12:25 +12:00
Jimmy Wärting 870be1e2d5 Use a WeakMap for private properties (#42)
* options.buffer isn't a valid option

* truly private internal properties

* 100% test coverage
2020-06-06 00:09:32 -04:00
Antoni Kepinski 960f7e528d Merge branch 'master' of https://github.com/node-fetch/fetch-blob 2020-05-21 11:08:12 +02:00
Antoni Kepinski 582880a297 bump version number 2020-05-21 11:08:09 +02:00
Antoni Kepinski c1dd0851d8 Update CHANGELOG.md 2020-05-21 11:07:56 +02:00
Linus Unnebäck bec5a3dceb Use upstream Blob directly in typings (#38) 2020-05-21 09:12:04 +01:00
Richie Bendall 6fabfce52f Update dependencies
Signed-off-by: Richie Bendall <richiebendall@gmail.com>
2020-05-19 21:59:02 +12:00
Richie Bendall 4a46def452 Maintenance update
Signed-off-by: Richie Bendall <richiebendall@gmail.com>
2020-04-14 21:52:25 +12:00
Richie Bendall 9a458af871 Remove lockfiles
Signed-off-by: Richie Bendall <richiebendall@gmail.com>
2020-04-14 21:47:17 +12:00
Richie Bendall f7e60156a4 docs: Fix grammar
Signed-off-by: Richie Bendall <richiebendall@gmail.com>
2020-03-22 16:14:38 +13:00
Richie Bendall af2edbd01c Merge branch 'master' of https://github.com/node-fetch/fetch-blob 2020-03-22 15:45:43 +13:00
Richie Bendall f4ddd159a4 docs: Update meta
Signed-off-by: Richie Bendall <richiebendall@gmail.com>
2020-03-22 15:45:40 +13:00
dependabot-preview[bot]anddependabot-preview[bot] <27856297+dependabot-preview[bot]@users.noreply.github.com> 83559d1144 [Security] Bump acorn from 7.1.0 to 7.1.1 (#23)
Bumps [acorn](https://github.com/acornjs/acorn) from 7.1.0 to 7.1.1. **This update includes a security fix.**
- [Release notes](https://github.com/acornjs/acorn/releases)
- [Commits](https://github.com/acornjs/acorn/compare/7.1.0...7.1.1)

Signed-off-by: dependabot-preview[bot] <support@dependabot.com>

Co-authored-by: dependabot-preview[bot] <27856297+dependabot-preview[bot]@users.noreply.github.com>
2020-03-22 15:43:12 +13:00
dependabot-preview[bot] 8bf6441343 Bump codecov from 3.6.1 to 3.6.5 (#14)
Bumps [codecov](https://github.com/codecov/codecov-node) from 3.6.1 to 3.6.5.
- [Release notes](https://github.com/codecov/codecov-node/releases)
- [Commits](https://github.com/codecov/codecov-node/commits)

Signed-off-by: dependabot-preview[bot] <support@dependabot.com>
2020-02-25 11:13:37 +13:00
Richie Bendall 1f5258cd79 chore: Update dependencies
Signed-off-by: Richie Bendall <richiebendall@gmail.com>
2020-02-13 21:48:24 +13:00
Richie Bendall f336a8d182 feat: Add typings (#6)
Signed-off-by: Richie Bendall <richiebendall@gmail.com>
2019-12-15 19:35:42 +08:00
David Frank b09fdac531 v1.0.5
* update dev dependency

* changelog update

* 1.0.5
2019-11-24 00:02:18 +08:00
14 changed files with 834 additions and 5627 deletions
+6
View File
@@ -0,0 +1,6 @@
coverage:
round: up
precision: 2
parsers:
javascript:
enable_partials: yes
+56
View File
@@ -0,0 +1,56 @@
name: CI
on:
push:
branches: [master]
pull_request:
paths:
- "**.js"
- "package.json"
- ".github/workflows/ci.yml"
jobs:
test:
strategy:
matrix:
os: [ubuntu-latest, windows-latest, macOS-latest]
node: ["16", "15", "14", engines]
exclude:
# On Windows, run tests with only the LTS environments.
- os: windows-latest
node: engines
- os: windows-latest
node: "14"
# On macOS, run tests with only the LTS environments.
- os: macOS-latest
node: engines
- os: macOS-latest
node: "14"
runs-on: ${{ matrix.os }}
steps:
- uses: actions/checkout@v2
- name: Get Node.JS version from package.json
if: matrix.node == 'engines'
id: get-version
run: echo ::set-output name=node::$(npx --q minimum-node-version)
- uses: actions/setup-node@v2-beta
if: matrix.node != 'engines'
with:
node-version: ${{ matrix.node }}
- uses: actions/setup-node@v2-beta
if: matrix.node == 'engines'
with:
node-version: ${{steps.get-version.outputs.node}}
- run: npm install
- run: npm run report -- --colors
- name: Upload coverage to Codecov
uses: codecov/codecov-action@v1
with:
file: ./coverage/coverage-final.json
+20
View File
@@ -0,0 +1,20 @@
name: CI
on:
pull_request:
paths:
- "**.js"
- "**eslint**"
- "package.json"
- ".github/workflows/lint.yml"
jobs:
lint:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v2
- name: Use Node.js
uses: actions/setup-node@v2-beta
with:
node-version: 14
- run: npm install
- run: npm run lint
+7 -2
View File
@@ -1,3 +1,8 @@
# Lock files
package-lock.json
yarn.lock
# Logs
logs
*.log
@@ -57,5 +62,5 @@ typings/
# dotenv environment variables file
.env
# next.js build output
.next
index.d.ts
from.d.ts
+2
View File
@@ -0,0 +1,2 @@
package-lock=false
save-exact=false
-11
View File
@@ -1,11 +0,0 @@
language: node_js
node_js:
- "8"
- "10"
- "12"
- "node"
script:
- npm run coverage
cache:
directories:
- node_modules
+50 -2
View File
@@ -1,10 +1,58 @@
Changelog
=========
## v3.0.0-rc.0
- Changed WeakMap for private field (require node 12)
- Switch to ESM
- blob.stream() return a subset of whatwg stream which is the async iterable
(it no longer return a node stream)
- Reduced the dependency of Buffer by changing to global TextEncoder/Decoder (require node 11)
- Disabled xo since it could understand private fields (#)
- No longer transform the type to lowercase (https://github.com/w3c/FileAPI/issues/43)
This is more loose than strict, keys should be lowercased, but values should not.
It would require a more proper mime type parser - so we just made it loose.
- index.js can now be imported by browser & deno since it no longer depends on any
core node features (but why would you? other environment can benefit from it)
## v2.1.2
- Fixed a bug where `start` in BlobDataItem was undefined (#85)
## v2.1.1
- Add nullish values checking in Symbol.hasInstance (#82)
- Add generated typings for from.js file (#80)
- Updated dev dependencies
## v2.1.0
- Fix: .slice has an implementation bug (#54).
- Added blob backed up by filesystem (#55)
## v2.0.1
- Fix: remove upper bound for node engine semver (#49).
## v2.0.0
> Note: This release was previously published as `1.0.7`, but as it contains breaking changes, we renamed it to `2.0.0`.
- **Breaking:** minimum supported Node.js version is now 10.17.
- **Breaking:** `buffer` option has been removed.
- Enhance: create TypeScript declarations from JSDoc (#45).
- Enhance: operate on blob parts (byte sequence) (#44).
- Enhance: use a `WeakMap` for private properties (#42) .
- Other: update formatting.
## v1.0.6
- Enhance: use upstream Blob directly in typings (#38)
- Other: update dependencies
## v1.0.5
- Other: no change to code, update dev dependency to address vulnerability reports
## v1.0.4
- Other: general code revwrite to pass linting, prepare for `node-fetch` release v3
- Other: general code rewrite to pass linting, prepare for `node-fetch` release v3
## v1.0.3
+119 -19
View File
@@ -1,38 +1,138 @@
fetch-blob
==========
# fetch-blob
[![npm version][npm-image]][npm-url]
[![build status][travis-image]][travis-url]
[![build status][ci-image]][ci-url]
[![coverage status][codecov-image]][codecov-url]
[![install size][install-size-image]][install-size-url]
A Blob implementation on node.js, originally from node-fetch
A Blob implementation in Node.js, originally from [node-fetch](https://github.com/node-fetch/node-fetch).
## Installation
```sh
npm install fetch-blob
```
<details>
<summary>Upgrading from 2x to 3x</summary>
Updating from 2 to 3 should be a breeze since there is not many changes to the blob specification.
The major cause of a major release is coding standards.
- internal WeakMaps was replaced with private fields
- internal Buffer.from was replaced with TextEncoder/Decoder
- internal buffers was replaced with Uint8Arrays
- CommonJS was replaced with ESM
- The node stream returned by calling `blob.stream()` was replaced with a simple generator function that yields Uint8Array (Breaking change)
(Read "Differences from other blobs" for more info.)
All of this changes have made it dependency free of any core node modules, so it would be possible to just import it using http-import from a CDN without any bundling
</details>
<details>
<summary>Differences from other Blobs</summary>
- Unlike NodeJS `buffer.Blob` (Added in: v15.7.0) and browser native Blob this polyfilled version can't be sent via PostMessage
- This blob version is more arbitrary, it can be constructed with blob parts that isn't a instance of itself
it has to look and behave as a blob to be accepted as a blob part.
- The benefit of this is that you can create other types of blobs that don't contain any internal data that has to be read in other ways, such as the `BlobDataItem` created in `from.js` that wraps a file path into a blob-like item and read lazily (nodejs plans to [implement this][fs-blobs] as well)
- The `blob.stream()` is the most noticeable differences. It returns a AsyncGeneratorFunction that yields Uint8Arrays
The reasoning behind `Blob.prototype.stream()` is that NodeJS readable stream
isn't spec compatible with whatwg streams and we didn't want to import the hole whatwg stream polyfill for node
or browserify NodeJS streams for the browsers and picking any flavor over the other. So we decided to opted out
of any stream and just implement the bear minium of what both streams have in common which is the asyncIterator
that both yields Uint8Array. this is the most isomorphic way with the use of `for-await-of` loops.
It would be redundant to convert anything to whatwg streams and than convert it back to
node streams since you work inside of Node.
It will probably stay like this until nodejs get native support for whatwg<sup>[1][https://github.com/nodejs/whatwg-stream]</sup> streams and whatwg stream add the node
equivalent for `Readable.from(iterable)`<sup>[2](https://github.com/whatwg/streams/issues/1018)</sup>
But for now if you really need a Node Stream then you can do so using this transformation
```js
import {Readable} from 'stream'
const stream = Readable.from(blob.stream())
```
But if you don't need it to be a stream then you can just use the asyncIterator part of it that is isomorphic.
```js
for await (const chunk of blob.stream()) {
console.log(chunk) // uInt8Array
}
```
If you need to make some feature detection to fix this different behavior
```js
if (Blob.prototype.stream?.constructor?.name === 'AsyncGeneratorFunction') {
// not spec compatible, monkey patch it...
// (Alternative you could extend the Blob and use super.stream())
let orig = Blob.prototype.stream
Blob.prototype.stream = function () {
const iterator = orig.call(this)
return new ReadableStream({
async pull (ctrl) {
const next = await iterator.next()
return next.done ? ctrl.close() : ctrl.enqueue(next.value)
}
})
}
}
```
Possible feature whatwg version: `ReadableStream.from(iterator)`
It's also possible to delete this method and instead use `.slice()` and `.arrayBuffer()` since it has both a public and private stream method
</details>
## Usage
```js
// Ways to import
// (PS it's dependency free ESM package so regular http-import from CDN works too)
import Blob from 'fetch-blob'
import {Blob} from 'fetch-blob'
const {Blob} = await import('fetch-blob')
// Ways to read the blob:
const blob = new Blob(['hello, world'])
await blob.text()
await blob.arrayBuffer()
for await (let chunk of blob.stream()) { ... }
// turn the async iterator into a node stream
stream.Readable.from(blob.stream())
// turn the async iterator into a whatwg stream (feature)
globalThis.ReadableStream.from(blob.stream())
```
### Blob part backed up by filesystem
To use, install [domexception](https://github.com/jsdom/domexception).
```sh
$ npm install fetch-blob
npm install fetch-blob domexception
```
```js
const Blob = require('fetch-blob');
const fetch = require('node-fetch');
fetch('https://httpbin.org/post', {
method: 'POST',
body: new Blob(['hello'], { type: 'text/plain' })
})
.then(res => res.json()) // expecting a json response
.then(json => console.log(json));
// The default export is sync and use fs.stat to retrieve size & last modified
import blobFromSync from 'fetch-blob/from.js'
import {Blob, blobFrom, blobFromSync} from 'fetch-blob/from.js'
const fsBlob1 = blobFromSync('./2-GiB-file.bin')
const fsBlob2 = await blobFrom('./2-GiB-file.bin')
// Not a 4 GiB memory snapshot, just holds 3 references
// points to where data is located on the disk
const blob = new Blob([fsBlob1, fsBlob2, 'memory'])
console.log(blob.size) // 4 GiB
```
See [MDN doc](https://developer.mozilla.org/en-US/docs/Web/API/Blob) and [Tests](https://github.com/bitinn/fetch-blob/blob/master/test.js) for more details.
See the [MDN documentation](https://developer.mozilla.org/en-US/docs/Web/API/Blob) and [tests](https://github.com/node-fetch/fetch-blob/blob/master/test.js) for more details.
[npm-image]: https://flat.badgen.net/npm/v/fetch-blob
[npm-url]: https://www.npmjs.com/package/fetch-blob
[travis-image]: https://flat.badgen.net/travis/bitinn/fetch-blob
[travis-url]: https://travis-ci.org/bitinn/fetch-blob
[codecov-image]: https://flat.badgen.net/codecov/c/github/bitinn/fetch-blob/master
[codecov-url]: https://codecov.io/gh/bitinn/fetch-blob
[ci-image]: https://github.com/node-fetch/fetch-blob/workflows/CI/badge.svg
[ci-url]: https://github.com/node-fetch/fetch-blob/actions
[codecov-image]: https://flat.badgen.net/codecov/c/github/node-fetch/fetch-blob/master
[codecov-url]: https://codecov.io/gh/node-fetch/fetch-blob
[install-size-image]: https://flat.badgen.net/packagephobia/install/fetch-blob
[install-size-url]: https://packagephobia.now.sh/result?p=fetch-blob
[fs-blobs]: https://github.com/nodejs/node/issues/37340
-135
View File
@@ -1,135 +0,0 @@
// Based on https://github.com/tmpvar/jsdom/blob/aa85b2abf07766ff7bf5c1f6daafb3726f2f2db5/lib/jsdom/living/blob.js
// (MIT licensed)
const Stream = require('stream');
// Fix for "Readable" isn't a named export issue
const {Readable} = Stream;
const BUFFER = Symbol('buffer');
const TYPE = Symbol('type');
class Blob {
constructor(...args) {
this[TYPE] = '';
const blobParts = args[0];
const options = args[1];
const buffers = [];
/* eslint-disable-next-line no-unused-vars */
let size = 0;
if (blobParts) {
blobParts.forEach(element => {
let buffer;
if (element instanceof Buffer) {
buffer = element;
} else if (ArrayBuffer.isView(element)) {
buffer = Buffer.from(element.buffer, element.byteOffset, element.byteLength);
} else if (element instanceof ArrayBuffer) {
buffer = Buffer.from(element);
} else if (element instanceof Blob) {
buffer = element[BUFFER];
} else {
buffer = Buffer.from(typeof element === 'string' ? element : String(element));
}
size += buffer.length;
buffers.push(buffer);
});
}
this[BUFFER] = Buffer.concat(buffers);
const type = options && options.type !== undefined && String(options.type).toLowerCase();
if (type && !/[^\u0020-\u007E]/.test(type)) {
this[TYPE] = type;
}
if (options && Buffer.isBuffer(options.buffer)) {
this[BUFFER] = options.buffer;
}
}
get size() {
return this[BUFFER].length;
}
get type() {
return this[TYPE];
}
text() {
return Promise.resolve(this[BUFFER].toString());
}
arrayBuffer() {
const buf = this[BUFFER];
const ab = buf.buffer.slice(buf.byteOffset, buf.byteOffset + buf.byteLength);
return Promise.resolve(ab);
}
stream() {
const readable = new Readable();
readable._read = () => { };
readable.push(this[BUFFER]);
readable.push(null);
return readable;
}
toString() {
return '[object Blob]';
}
slice(...args) {
const {size} = this;
const start = args[0];
const end = args[1];
let relativeStart;
let relativeEnd;
if (start === undefined) {
relativeStart = 0;
} else if (start < 0) {
relativeStart = Math.max(size + start, 0);
} else {
relativeStart = Math.min(start, size);
}
if (end === undefined) {
relativeEnd = size;
} else if (end < 0) {
relativeEnd = Math.max(size + end, 0);
} else {
relativeEnd = Math.min(end, size);
}
const span = Math.max(relativeEnd - relativeStart, 0);
const buffer = this[BUFFER];
const slicedBuffer = buffer.slice(
relativeStart,
relativeStart + span
);
const blob = new Blob([], {type: args[2]});
blob[BUFFER] = slicedBuffer;
return blob;
}
}
Object.defineProperties(Blob.prototype, {
size: {enumerable: true},
type: {enumerable: true},
slice: {enumerable: true}
});
Object.defineProperty(Blob.prototype, Symbol.toStringTag, {
value: 'Blob',
writable: false,
enumerable: false,
configurable: true
});
module.exports = Blob;
+75
View File
@@ -0,0 +1,75 @@
import {statSync, createReadStream} from 'fs';
import {stat} from 'fs/promises';
import DOMException from 'domexception';
import Blob from './index.js';
/**
* @param {string} path filepath on the disk
* @returns {Blob}
*/
const blobFromSync = path => from(statSync(path), path);
/**
* @param {string} path filepath on the disk
* @returns {Promise<Blob>}
*/
const blobFrom = path => stat(path).then(stat => from(stat, path));
const from = (stat, path) => new Blob([new BlobDataItem({
path,
size: stat.size,
lastModified: stat.mtimeMs,
start: 0
})]);
/**
* This is a blob backed up by a file on the disk
* with minium requirement. Its wrapped around a Blob as a blobPart
* so you have no direct access to this.
*
* @private
*/
class BlobDataItem {
#path;
#start;
constructor(options) {
this.#path = options.path;
this.#start = options.start;
this.size = options.size;
this.lastModified = options.lastModified
}
/**
* Slicing arguments is first validated and formatted
* to not be out of range by Blob.prototype.slice
*/
slice(start, end) {
return new BlobDataItem({
path: this.#path,
lastModified: this.lastModified,
size: end - start,
start
});
}
async * stream() {
const {mtimeMs} = await stat(this.#path)
if (mtimeMs > this.lastModified) {
throw new DOMException('The requested file could not be read, typically due to permission problems that have occurred after a reference to a file was acquired.', 'NotReadableError');
}
if (this.size) {
yield * createReadStream(this.#path, {
start: this.#start,
end: this.#start + this.size - 1
});
}
}
get [Symbol.toStringTag]() {
return 'Blob';
}
}
export default blobFromSync;
export {Blob, blobFrom, blobFromSync};
+216
View File
@@ -0,0 +1,216 @@
// 64 KiB (same size chrome slice theirs blob into Uint8array's)
const POOL_SIZE = 65536;
/** @param {(Blob | Uint8Array)[]} parts */
async function * toIterator (parts, clone = true) {
for (let part of parts) {
if ('stream' in part) {
yield * part.stream();
} else if (ArrayBuffer.isView(part)) {
if (clone) {
let position = part.byteOffset;
let end = part.byteOffset + part.byteLength;
while (position !== end) {
const size = Math.min(end - position, POOL_SIZE);
const chunk = part.buffer.slice(position, position + size);
yield new Uint8Array(chunk);
position += chunk.byteLength;
}
} else {
yield part;
}
} else {
// For blobs that have arrayBuffer but no stream method (nodes buffer.Blob)
let position = 0;
while (position !== part.size) {
const chunk = part.slice(position, Math.min(part.size, position + POOL_SIZE));
const buffer = await chunk.arrayBuffer();
position += buffer.byteLength;
yield new Uint8Array(buffer);
}
}
}
}
export default class Blob {
/** @type {Array.<(Blob|Uint8Array)>} */
#parts = [];
#type = '';
#size = 0;
/**
* The Blob() constructor returns a new Blob object. The content
* of the blob consists of the concatenation of the values given
* in the parameter array.
*
* @param {*} blobParts
* @param {{ type?: string }} [options]
*/
constructor(blobParts = [], options = {}) {
let size = 0;
const parts = blobParts.map(element => {
let part;
if (ArrayBuffer.isView(element)) {
part = new Uint8Array(element.buffer.slice(element.byteOffset, element.byteOffset + element.byteLength));
} else if (element instanceof ArrayBuffer) {
part = new Uint8Array(element.slice(0));
} else if (element instanceof Blob) {
part = element;
} else {
part = new TextEncoder().encode(String(element));
}
size += ArrayBuffer.isView(part) ? part.byteLength : part.size;
return part;
});
const type = options.type === undefined ? '' : String(options.type);
this.#type = /[^\u0020-\u007E]/.test(type) ? '' : type;
this.#size = size;
this.#parts = parts;
}
/**
* The Blob interface's size property returns the
* size of the Blob in bytes.
*/
get size() {
return this.#size;
}
/**
* The type property of a Blob object returns the MIME type of the file.
*/
get type() {
return this.#type;
}
/**
* The text() method in the Blob interface returns a Promise
* that resolves with a string containing the contents of
* the blob, interpreted as UTF-8.
*
* @return {Promise<string>}
*/
async text() {
// More optimized than using this.arrayBuffer()
// that requires twice as much ram
const decoder = new TextDecoder();
let str = '';
for await (let part of toIterator(this.#parts, false)) {
str += decoder.decode(part, { stream: true });
}
// Remaining
str += decoder.decode();
return str;
}
/**
* The arrayBuffer() method in the Blob interface returns a
* Promise that resolves with the contents of the blob as
* binary data contained in an ArrayBuffer.
*
* @return {Promise<ArrayBuffer>}
*/
async arrayBuffer() {
const data = new Uint8Array(this.size);
let offset = 0;
for await (const chunk of toIterator(this.#parts, false)) {
data.set(chunk, offset);
offset += chunk.length;
}
return data.buffer;
}
/**
* The Blob stream() implements partial support of the whatwg stream
* by only being async iterable.
*
* @returns {AsyncGenerator<Uint8Array>}
*/
async * stream() {
yield * toIterator(this.#parts, true);
}
/**
* The Blob interface's slice() method creates and returns a
* new Blob object which contains data from a subset of the
* blob on which it's called.
*
* @param {number} [start]
* @param {number} [end]
* @param {string} [type]
*/
slice(start = 0, end = this.size, type = '') {
const {size} = this;
let relativeStart = start < 0 ? Math.max(size + start, 0) : Math.min(start, size);
let relativeEnd = end < 0 ? Math.max(size + end, 0) : Math.min(end, size);
const span = Math.max(relativeEnd - relativeStart, 0);
const parts = this.#parts;
const blobParts = [];
let added = 0;
for (const part of parts) {
const size = ArrayBuffer.isView(part) ? part.byteLength : part.size;
if (relativeStart && size <= relativeStart) {
// Skip the beginning and change the relative
// start & end position as we skip the unwanted parts
relativeStart -= size;
relativeEnd -= size;
} else {
let chunk
if (ArrayBuffer.isView(part)) {
chunk = part.subarray(relativeStart, Math.min(size, relativeEnd));
added += chunk.byteLength
} else {
chunk = part.slice(relativeStart, Math.min(size, relativeEnd));
added += chunk.size
}
blobParts.push(chunk);
relativeStart = 0; // All next sequental parts should start at 0
// don't add the overflow to new blobParts
if (added >= span) {
break;
}
}
}
const blob = new Blob([], {type: String(type).toLowerCase()});
blob.#size = span;
blob.#parts = blobParts;
return blob;
}
get [Symbol.toStringTag]() {
return 'Blob';
}
static [Symbol.hasInstance](object) {
return (
object &&
typeof object === 'object' &&
typeof object.constructor === 'function' &&
(
typeof object.stream === 'function' ||
typeof object.arrayBuffer === 'function'
) &&
/^(Blob|File)$/.test(object[Symbol.toStringTag])
);
}
}
Object.defineProperties(Blob.prototype, {
size: {enumerable: true},
type: {enumerable: true},
slice: {enumerable: true}
});
export { Blob };
-5381
View File
File diff suppressed because it is too large Load Diff
+67 -30
View File
@@ -1,32 +1,69 @@
{
"name": "fetch-blob",
"version": "1.0.4",
"description": "A Blob implementation on node.js, originally from node-fetch",
"main": "blob.js",
"scripts": {
"test": "ava",
"report": "nyc ava",
"coverage": "nyc --reporter json --reporter text ava && codecov -f coverage/coverage-final.json"
},
"repository": {
"type": "git",
"url": "git+https://github.com/bitinn/fetch-blob.git"
},
"keywords": [
"blob",
"node-fetch"
],
"author": "David Frank",
"license": "MIT",
"bugs": {
"url": "https://github.com/bitinn/fetch-blob/issues"
},
"homepage": "https://github.com/bitinn/fetch-blob#readme",
"devDependencies": {
"ava": "^1.4.1",
"codecov": "^3.5.0",
"get-stream": "^5.1.0",
"node-fetch": "^2.6.0",
"nyc": "^14.1.1"
}
"name": "fetch-blob",
"version": "3.0.0-rc.0",
"description": "A Blob implementation in Node.js, originally from node-fetch.",
"main": "index.js",
"type": "module",
"files": [
"from.js",
"index.js",
"index.d.ts",
"from.d.ts"
],
"scripts": {
"lint": "xo test.js",
"test": "npm run lint && ava",
"report": "c8 --reporter json --reporter text ava",
"coverage": "c8 --reporter json --reporter text ava && codecov -f coverage/coverage-final.json",
"prepublishOnly": "tsc --declaration --emitDeclarationOnly --allowJs index.js from.js"
},
"repository": "https://github.com/node-fetch/fetch-blob.git",
"keywords": [
"blob",
"node-fetch"
],
"engines": {
"node": ">=14.0.0"
},
"author": "David Frank",
"license": "MIT",
"bugs": {
"url": "https://github.com/node-fetch/fetch-blob/issues"
},
"homepage": "https://github.com/node-fetch/fetch-blob#readme",
"xo": {
"rules": {
"unicorn/import-index": "off",
"import/extensions": [
"error",
"always",
{
"ignorePackages": true
}
]
},
"overrides": [
{
"files": "test.js",
"rules": {
"node/no-unsupported-features/es-syntax": 0,
"node/no-unsupported-features/node-builtins": 0
}
}
]
},
"peerDependenciesMeta": {
"domexception": {
"optional": true
}
},
"devDependencies": {
"ava": "^3.15.0",
"c8": "^7.7.1",
"codecov": "^3.8.1",
"domexception": "^2.0.1",
"node-fetch": "^3.0.0-beta.9",
"typescript": "^4.2.4",
"xo": "^0.38.2"
}
}
+216 -47
View File
@@ -1,87 +1,256 @@
import fs from 'fs';
import test from 'ava';
import Blob from './blob';
import util from 'util';
import stream from 'stream';
import getStream from 'get-stream';
import { Response } from 'node-fetch';
import {Response} from 'node-fetch';
import {Readable} from 'stream';
import buffer from 'buffer';
import Blob from './index.js';
import syncBlob, {blobFromSync, blobFrom} from './from.js';
const TextDecoder = util.TextDecoder;
const Readable = stream.Readable;
const license = fs.readFileSync('./LICENSE', 'utf-8');
test('Blob ctor', t => {
let data = 'a=1';
let blob = new Blob([data]);
test('new Blob()', t => {
const blob = new Blob(); // eslint-disable-line no-unused-vars
t.pass();
});
test('new Blob(parts)', t => {
const data = 'a=1';
const blob = new Blob([data]); // eslint-disable-line no-unused-vars
t.pass();
});
test('Blob ctor parts', async t => {
const parts = [
'a',
new Uint8Array([98]),
new Uint16Array([25699]),
new Uint8Array([101]).buffer,
Buffer.from('f'),
new Blob(['g']),
{},
new URLSearchParams('foo')
];
const blob = new Blob(parts);
t.is(await blob.text(), 'abcdefg[object Object]foo=');
});
test('Blob size', t => {
let data = 'a=1';
let blob = new Blob([data]);
const data = 'a=1';
const blob = new Blob([data]);
t.is(blob.size, data.length);
});
test('Blob type', t => {
let data = 'a=1';
let type = 'text/plain';
let blob = new Blob([data], { type });
const type = 'text/plain';
const blob = new Blob([], {type});
t.is(blob.type, type);
});
test('Blob slice type', t => {
const type = 'text/plain';
const blob = new Blob().slice(0, 0, type);
t.is(blob.type, type);
});
test('invalid Blob type', t => {
const blob = new Blob([], {type: '\u001Ftext/plain'});
t.is(blob.type, '');
});
test('invalid Blob slice type', t => {
const blob = new Blob().slice(0, 0, '\u001Ftext/plain');
t.is(blob.type, '');
});
test('Blob text()', async t => {
let data = 'a=1';
let type = 'text/plain';
let blob = new Blob([data], { type });
const data = 'a=1';
const type = 'text/plain';
const blob = new Blob([data], {type});
t.is(await blob.text(), data);
});
test('Blob arrayBuffer()', async t => {
let data = 'a=1';
let type = 'text/plain';
let blob = new Blob([data], { type });
const data = 'a=1';
const type = 'text/plain';
const blob = new Blob([data], {type});
let decoder = new TextDecoder('utf-8');
let buffer = await blob.arrayBuffer();
const decoder = new TextDecoder('utf-8');
const buffer = await blob.arrayBuffer();
t.is(decoder.decode(buffer), data);
});
test('Blob stream()', async t => {
let data = 'a=1';
let type = 'text/plain';
let blob = new Blob([data], { type });
let result = await getStream(blob.stream());
t.is(result, data);
const data = 'a=1';
const type = 'text/plain';
const blob = new Blob([data], {type});
for await (const chunk of blob.stream()) {
t.is(chunk.join(), [97, 61, 49].join());
}
});
test('Blob toString()', t => {
let data = 'a=1';
let type = 'text/plain';
let blob = new Blob([data], { type });
const data = 'a=1';
const type = 'text/plain';
const blob = new Blob([data], {type});
t.is(blob.toString(), '[object Blob]');
});
test('Blob slice()', async t => {
let data = 'a=1';
let type = 'text/plain';
let blob = new Blob([data], { type });
let blob2 = blob.slice(0, 1);
t.is(await blob2.text(), data.slice(0, 1));
const data = 'abcdefgh';
const blob = new Blob([data]).slice();
t.is(await blob.text(), data);
});
test('Blob slice(0, 1)', async t => {
const data = 'abcdefgh';
const blob = new Blob([data]).slice(0, 1);
t.is(await blob.text(), 'a');
});
test('Blob slice(-1)', async t => {
const data = 'abcdefgh';
const blob = new Blob([data]).slice(-1);
t.is(await blob.text(), 'h');
});
test('Blob slice(0, -1)', async t => {
const data = 'abcdefgh';
const blob = new Blob([data]).slice(0, -1);
t.is(await blob.text(), 'abcdefg');
});
test('Blob(["hello ", "world"]).slice(5)', async t => {
const parts = ['hello ', 'world'];
const blob = new Blob(parts);
t.is(await blob.slice(5).text(), ' world');
});
test('throw away unwanted parts', async t => {
const blob = new Blob(['a', 'b', 'c']).slice(1, 2);
t.is(await blob.text(), 'b');
});
test('Blob works with node-fetch Response.blob()', async t => {
let data = 'a=1';
let type = 'text/plain';
let blob = new Blob([data], { type });
let res = new Response(blob);
let blob2 = await res.blob();
const data = 'a=1';
const type = 'text/plain';
const blob = new Blob([data], {type});
const response = new Response(Readable.from(blob.stream()));
const blob2 = await response.blob();
t.is(await blob2.text(), data);
});
test('Blob works with node-fetch Response.text()', async t => {
let data = 'a=1';
let type = 'text/plain';
let blob = new Blob([data], { type });
let res = new Response(blob);
let text = await res.text();
const data = 'a=1';
const type = 'text/plain';
const blob = new Blob([data], {type});
const response = new Response(Readable.from(blob.stream()));
const text = await response.text();
t.is(text, data);
});
test('blob part backed up by filesystem', async t => {
const blob = blobFromSync('./LICENSE');
t.is(await blob.slice(0, 3).text(), license.slice(0, 3));
t.is(await blob.slice(4, 11).text(), license.slice(4, 11));
});
test('Reading after modified should fail', async t => {
const blob = blobFromSync('./LICENSE');
await new Promise(resolve => {
setTimeout(resolve, 100);
});
const now = new Date();
// Change modified time
fs.utimesSync('./LICENSE', now, now);
const error = await blob.text().catch(error => error);
t.is(error instanceof Error, true);
t.is(error.name, 'NotReadableError');
});
test('Reading from the stream created by blobFrom', async t => {
const blob = blobFromSync('./LICENSE');
const actual = await blob.text();
t.is(actual, license);
});
test('create a blob from path asynchronous', async t => {
const blob = await blobFrom('./LICENSE');
const actual = await blob.text();
t.is(actual, license);
});
test('Reading empty blobs', async t => {
const blob = blobFromSync('./LICENSE').slice(0, 0);
const actual = await blob.text();
t.is(actual, '');
});
test('Blob-ish class is an instance of Blob', t => {
class File {
stream() {}
get [Symbol.toStringTag]() {
return 'File';
}
}
t.true(new File() instanceof Blob);
});
test('Instanceof check returns false for nullish values', t => {
t.false(null instanceof Blob);
});
/** @see https://github.com/w3c/FileAPI/issues/43 - important to keep boundary value */
test('Dose not lowercase the blob values', t => {
const type = 'multipart/form-data; boundary=----WebKitFormBoundaryTKqdrVt01qOBltBd';
t.is(new Blob([], {type}).type, type);
});
test('Parts are immutable', async t => {
const buf = new Uint8Array([97]);
const blob = new Blob([buf]);
buf[0] = 98;
t.is(await blob.text(), 'a');
});
test('Blobs are immutable', async t => {
const buf = new Uint8Array([97]);
const blob = new Blob([buf]);
const chunk = await blob.stream().next();
t.is(chunk.value[0], 97);
chunk.value[0] = 98;
t.is(await blob.text(), 'a');
});
// This was necessary to avoid large ArrayBuffer clones (slice)
test('Large chunks are divided into smaller chunks', async t => {
const buf = new Uint8Array(65590);
const blob = new Blob([buf]);
let i = 0;
// eslint-disable-next-line no-unused-vars
for await (const chunk of blob.stream()) {
i++;
}
t.is(i === 2, true);
});
test('Can use named import - as well as default', async t => {
const {Blob, default: def} = await import('./index.js');
t.is(Blob, def);
});
test('default from.js exports blobFromSync', t => {
t.is(blobFromSync, syncBlob);
});
if (buffer.Blob) {
test('Can wrap buffer.Blob to a fetch-blob', async t => {
const blob1 = new buffer.Blob(['blob part']);
const blob2 = new Blob([blob1]);
t.is(await blob2.text(), 'blob part');
});
}