Blazor : how to use OpenCV in a component
Since v3.3.1, OpenCV is available for javascript. It is actually built from LLVM (c++) binaries thanks to emscripten. The releases are available here.
Script size
If you look at the size of the different versions of opencv.js, you'll see that it ranges from ~4Mb to 9Mb for v4.8.0. Depending on what you're trying to achieve, picking a version that covers your need for a minimal size is imo the recommended approach. However if you don't care about memory usage or loading times, then go for the latest stable version.
Recompiling the library
You always have the option to recompile the script by disabling the modules that you don't need which could cut the size in half (especially by disabling -DBUILD_opencv_dnn). It could be a good compromise between having the latest version but limited to your application's need.
Precautions:
- At this day you can't build opencv.js from a Windows environment (unless you use a linux distro in WSL)
- In my experience, the built image had a flaw, well explained here. My js component wouldn't work with the compiled file whereas it worked fine with the official release. Didn't dig deeper into this issue, but if you have weird errors like
right-hand side of 'instanceof' is not an object, consider trying with the official script.
The recompiling steps are documented in official documentation. I had no trouble to build it, just used the following command instead of the one provided in docs (from opencv repo's root directory):
python platforms/js/build_js.py build_out --emscripten_dir ../emsdk/upstream/emscripten --build_wasm --clean_build_dir
Adding it to Blazor
Statically
If you read the docs, you simply need to add the following lines in your index.html file to start using OpenCV in your javascript modules:
<script src="_content/MyComp/opencv.js" type="text/javascript"></script>
Note: This example supposes that your opencv.js file is under wwwroot in your component's project.
And well, here start the troubles. The following error messages will occasionnally happen when you start your software:
TypeError: Failed to execute 'compile' on 'WebAssembly': Cannot compile WebAssembly.Module from an already read ResponseModule.ready couldn't be redefined
For someone new to webassmbly and Blazor, these errors are really not helpful. After reading a bit on emscripten and after several dry attempts to find a solution online, here is what I finally did to make it work. Open your opencv.js file and replace the last lines as shown below:
if (typeof Module === 'undefined')
Module = {};
return cv(Module);
}));
with
if (typeof OpenCvModule === 'undefined')
OpenCvModule = {};
return cv(OpenCvModule);
}));
My understanding is that emscripten creates a Module variable in the global space by default when compiling a js file. Since _framework/blazor.webassembly.js also defines Module, the runtime complains. Renaming the variable fixes the issue. There is probably a cleaner way of resolving this issue and I'll update this article accordingly when I get more familiar with emscripten and webassembly in general.
Dynamically
If you only want to load opencv when needed, you need to load it dynamically from javascript. Copy paste the following function in your typescript class (or convert it to a function) :
private initOpenCv() {
if (typeof cv === 'undefined') {
// OpenCV.js not loaded yet, load it asynchronously
let script = document.createElement('script');
script.setAttribute('async', '');
script.setAttribute('type', 'text/javascript');
script.addEventListener('load', async () => {
// Checking the loading of the buildInformation
if (cv instanceof Promise) {
cv = await cv;
console.info("OpenCv ready");
console.log(cv.getBuildInformation());
this._isOpenCvRuntimeReady = true;
}
else if (cv != 'undefined' && cv.getBuildInformation) {
console.info("OpenCv ready");
console.log(cv.getBuildInformation());
}
else {
// Web Assembly check
cv['onRuntimeInitialized'] = () => {
console.info("OpenCv ready");
console.log(cv.getBuildInformation());
this._isOpenCvRuntimeReady = true;
}
}
});
script.addEventListener('error', () => {
console.error('Failed to load opencv');
});
script.src = '_content/MyComp/opencv.js';
let node = document.getElementsByTagName('script')[0];
node.parentNode.insertBefore(script, node);
} else {
console.info("OpenCv already loaded");
this._isOpenCvRuntimeReady = true;
}
}
This script basically modifies the DOM to add the <script> tag on demand, loads it asynchronously and subscribes to the load event to detect the successful loading of opencv.
Call this method from a public init method of your class, or from your constructor.
Do not forget to set this._isOpenCvRuntimeReady to false in your constructor as well.
Other known issues
AMD vs ESM
If you somehow encounter the following error when loading your component :
Uncaught Error: Can only have one anonymous define call per script file
start by reading this article.
Your application has probably a mixup of technology in how javascript modularity is handled. In my case, the app was also using Microsoft's Monaco editor which comes with AMD loader by default. If you read the article carefully, you've seen that there's a pattern called UMD to verify the module system when importing a js library. OpenCV uses that pattern. In fact, commenting parts of the first line of opencv fixes the issue as shown below:
(function(root,factory){/*if(typeof define==='function'&&define.amd){define(function(){return(root.cv=factory());});}else */if(typeof module==='object'&&module.exports){...
but I have to tell you that this is an ugly fix because it breaks the UMD pattern (which is good !). You better review your js modules and make sure that they're aligned to the same module systems.