Introducing webassembly-loader: Yet another webpack-loader for importing wasm code 😂

in utopian-io •  last year 

Repository

https://github.com/DrSensor/webassembly-loader

About

Just as I mentioned when first releasing rs-jest, I need to create another wasm-loader alike because rs-jest and rollup-plugin-rust can export wasm into javascript module in various mode while wasm-loader and webpack webassembly/experimental don't have that feature.

future plans

This way rs-jest can be used in a project that either use Webpack or Rollup.

Current features

The feature are pretty much the same as rs-jest and rollup-plugin-rust export option. However, when used as a library, you can choose which module should be generated[ref]. Currently there is only 2 choices: CommonJS module (cjs) and ECMAScript module (esm).

Technology Stack

📦 Main Dependencies

The library that I use in this project is pretty much the same as described in official Webpack documentation[ref]. However, I have been in doubt[code] between using @webpack-contrib/schema-utils or schema-utils.

✔️ Test Framework/Approach

The test framework I used for this project is Jest running on Nodejs v8 or higher. The reason the minimum Nodejs is not v6 because the WebAssembly support begins in Nodejs v8. For the approach, I implement 2 kinds of test: unit-tests and smoke-tests. The unit-tests use Jest which automatically confirms that the generated code can be run in Nodejs ≥8.x. Also, I choose to test the bundle results rather than test the source code which make https://codecov.io can't report the coverage (see #Others for the reason). For the smoke-tests itself, I choose a project that integrate Rust into Vue using rust-native-wasm-loader.

TestLinkenvironment
unithttps://github.com/DrSensor/webassembly-loader/tree/master/testnodejs
smokehttps://github.com/DrSensor/webassembly-loader/tree/master/examplesbrowser

Coverage results: PR#3

On the unit-tests, I split it into 3 test suites which represent the usages:

  • lib.test.ts will test if this package used as a library which the bundler/transpiler must respect pkg.module
  • loader.test.ts will test if this package used as a Webpack Loader
  • negative.test.ts will test all possible errors either when used as a library or as a Webpack Loader
♻️ CI and Build Tools

As usual, I use CircleCI for the automation build/testing. Unlike binaryen-loader, I don't structure the project base on webpack-defaults but rather on typescript-library-starter which use Rollup rather than Webpack. In the CI workflow itself, the difference with binaryen-loader is I disable the testing jobs for node6-latest and node8-canary (also add the smoke-test). For node8-canary, I disable it because there is no [email protected] since Webpack 4 was graduated from it's beta version :yay:

CI workflow on regular commit

CI workflow on tagging release
.

Ah yes, CI jobs draft and smoke-rust-vue require jobs node8-latest to be successfully run but seems CircleCI can't display it properly (the lines is overlap 😂)

💪 Others

As you can guess, I use Typescript for developing this webpack loader, either in the test scripts or in the source code. The interesting things when I use Typescript in this kind of setup is I need to split the configuration between bundling and testing because Jest doesn't support transpiling es6 import when importing CommonJS module (red: schema-utils and loader-utils). This is why I specify module code generation to CommonJS [code]. Thankfully, rollup-plugin-typescript2 has an option to override tsconfig.json configuration so it doesn't give me an extra headache 😂.

Future Plans

  • Need to figure out how to test the source code instead of the bundle results.
  • Add another export mode for map and async-map so that it can be used in Vue/Svelte projects seamlessly. Just like how we usually integrate Vuex in a Vue component, the syntax would be:
import { mapExports } from 'lib.rs'

export default {
 methods: {
     ...mapExports([
         '*cryptSHA3',
         'fourier_transform',
         'matrix_*'
     ])
 }
}

Something that I learn when creating this package

Webpack
  • Seems like there is an effort to make Webpack loader mechanism as a separate module [repo]. I hope it can make testing and supporting es6 export much more easier.
  • There is also an ongoing effort to make testing Webpack Loader/Plugin easier [repo]
Webassembly
  • There is a proposal for ES Module Integration [repo]
  • Rust v1.29.0 or later produce wasm file bigger than Rust v1.28.0. I found it when I try to compile simple add(a: i32, b: i32) operation targeting wasm32-unknown-unknown with LTO enabled and in release profile. I got ~57KB using rustc 1.29.2 and rustc 1.31.0-nightly while rustc 1.28.0 just produce 151 bytes. Someone from Rust discord server said that's because debuginfo is now always active and the only way to squash the size down is to run wasm-opt / wasm-gc and co. I'm sad.

How to contribute?

Just DM's me, make a Github issue or writes a comment below if someone interested. For guidelines, see Contributing section.

I don't plan to create parcel-plugin-rust since I never handle a project that use Parcel before. Feel free to contact me if anyone want to use this loader as a library. I'm willing to help out.

And...there is also a hacktoberfest task for this loader :p (I know I'm little bit late to join the hype).

GitHub Account

https://github.com/DrSensor


Authors get paid when people like you upvote their post.
If you enjoyed what you read here, create your account today and start earning FREE STEEM!
Sort Order:  

Thank you for your contribution. An awesome write-up, with a lot of learning. I really like your approach of writing test, +1 for that. Though I am not able to understand why have you created webassembly-loader when there are other webpack-loader, might be I am missing something.

I really like "Something that I learn when creating this package" section and I would say that everyone should share the same thing in their post which will benefit not only them but to the reader too.


Your contribution has been evaluated according to Utopian policies and guidelines, as well as a predefined set of questions pertaining to the category.

To view those questions and the relevant answers related to your post, click here.


Need help? Write a ticket on https://support.utopian.io/.
Chat with us on Discord.
[utopian-moderator]

Though I am not able to understand why have you created webassembly-loader when there are other webpack-loader

At first, I also have the same question about webpack 4 which begin to support webassembly file. why they add webassembly/experimental feature when there is wasm-loader?
I begin to understand the reason. Seems they do it for maintainability as well as experimenting some new technique on how to import a module.

Well, the reason I create another loader, as I said in the article:
This way rs-jest (a Jest transformer that I create to make Rust code can be used in Javascript project as a Webassembly module) can be used in a project that either uses Webpack or Rollup.
Summary, compatibility reason.

Another reason is I want to make maintaining "export mode" feature easier. Both in rs-jest and rollup-plugin-rust.

Thank you for your review, @codingdefined!

So far this week you've reviewed 2 contributions. Keep up the good work!

Hi @drsensor!

Your post was upvoted by @steem-ua, new Steem dApp, using UserAuthority for algorithmic post curation!
Your post is eligible for our upvote, thanks to our collaboration with @utopian-io!
Feel free to join our @steem-ua Discord server

Hey, @drsensor!

Thanks for contributing on Utopian.
We’re already looking forward to your next contribution!

Get higher incentives and support Utopian.io!
Simply set @utopian.pay as a 5% (or higher) payout beneficiary on your contribution post (via SteemPlus or Steeditor).

Want to chat? Join us on Discord https://discord.gg/h52nFrV.

Vote for Utopian Witness!