Unit Testing, Covering & Linting Your Power BI Custom Visual
This video explains how to set up Unit Testing, Code Coverage and Linting on your Power BI Custom Visual. This is video #20 of the Power BI Custom Visual Development Fundamentals series, please check the playlist below for the set.
Resources
Transcript
Hello again and welcome back to this series on developing Power BI Custom Visuals.
Today, I will show you show to setup unit testing, code coverage and linting on your custom visual project.
Unit Testing is crucial in keeping your code resilient to bugs and avoiding regression.
Code coverage helps in this process by pointing out what blocks of code are not being tested yet.
Linting on the other hand, points you at dodgy lines code that don’t follow best practices, and that even if compiling correctly, may be just asking for bugs under the right conditions.
In this video, I’ll show you how to setup a unit testing framework, a code coverage report and a linting process using Karma, Jasmine and tslint.
As a bonus, I’ll also show you how to configure your typescript project for custom build and test commands, making integration with a continuous build server a piece of cake.
So, let’s get to it, right now.
Today, I’m going to get you started with four new pieces of software.
The first one is Jasmine.
Jasmine a very popular unit testing framework for JavaScript.
It makes writing and maintaining unit tests very easy and it does not depend on any other frameworks.
The catch is that it doesn’t work with TypeScript out of the box and it does not support code coverage on its own.
To cover for that, pun intended, you will make use of Karma.
Karma is a very appropriately named test runner that is able to orchestrate different test frameworks and show an aggregated result of test runs and code coverage status.
When you install Karma, you also install Istanbul as a dependency.
Istanbul, named after the city is also a code coverage tool for JavaScript.
Thanks to this fella, you’ll get a report after every test run, telling you how much of your code is covered by unit tests.
The fourth package is tslint.
A linter is a tool that checks code for best practices, readability, maintainability and other issues that may cause bugs when you least expect.
Tslint does this job for typescript.
In addition to these four, there are some utility packages that you will need to install as well.
However, those will work more like glue to hold all this together and you will not interact with them yourself.
But anyway, let’s get to it.
I’ll start by installing all packages required.
For that, I’m going to open the package.json file.
This file lists all dependencies your project has.
Up until now I’ve only added production dependencies to this project.
These refer to dependencies that get packaged with your visual when you run pbiviz package.
However, when it comes to all the unit testing packages we’re going to install, you really only need these during development time.
You don’t want to package all this stuff into a production file as it will bloat the final product to no end.
Therefore, I’m going to install these packages as development dependencies.
This means the pbiviz package command will ignore them when packaging the final product while still letting me use everything during development time.
Now I’m also going to show you an alternate to install dependencies.
You’ve already seen that you can install extra packages by running npm install.
What you haven’t seen yet, if you’ve been following this series, is that you can also write the dependencies you require in this file here, and then request npm to install them for you.
And that’s I’m going to do now.
Instead of typing npm install 300 times, I’m just going to copy paste all unit testing dependencies from another project that I already know works.
"devDependencies": {
"@types/jasmine": "^2.5.54",
"@types/jasmine-jquery": "^1.5.32",
"@types/jquery": "^2.0.48",
"coveralls": "^2.13.1",
"jasmine": "^2.8.0",
"jasmine-core": "^2.8.0",
"jasmine-jquery": "^2.1.1",
"jasmine-ts": "^0.2.1",
"jquery": "^2.2.4",
"karma": "^1.7.0",
"karma-chrome-launcher": "^2.2.0",
"karma-coverage": "^1.1.1",
"karma-jasmine": "^1.1.0",
"karma-remap-istanbul": "^0.6.0",
"karma-sourcemap-loader": "^0.3.7",
"karma-typescript-preprocessor": "^0.3.1",
"lodash": "^4.17.4",
"powerbi-visuals-tools": "^1.7.2",
"powerbi-visuals-utils-testutils": "^1.0.1",
"ts-node": "^3.3.0",
"tslint": "^5.7.0",
"tslint-microsoft-contrib": "^5.0.1",
"typescript": "^2.4.2"
},
If you want to keep yourself from writing all this, you can just open this file from the github repository. In fact, I’m going to assume you’ll do exactly that.
Now notice how all these dependencies are inside this devDependencies block?
That’s what keeps the typescript compiler from including them in a production build, and therefore keeps pbiviz from packaging them in the final product.
In case you’re already using some of these packages in a build capacity, such as jquery, for example, you can take them out of this list.
There is no point duplicating dependencies between build and development.
Once you’ve copied these dependencies into your project, all you need to do is to run npm install on the terminal.
Npm will then go about its merry way and download any and all missing bits from the npm package repository.
In some rare cases, npm install may error out when you try to install many packages at the same.
When it does, it usually means one of two things.
- One, you probably have some old rubbish lying around in the node_modules folder that is somehow conflicting with the new dependencies.To sort that out run npm prune to delete all old stuff you no longer need.Then you can run npm install again.
- Now if that doesn’t work, then it means there is an incompatibility somewhere between all the latest versions you’re installing and any other dependencies you may already have.You can usually sort that out by installing these new dependencies one-by-one using the command line.Just remember to use –save-dev when running npm so you install them as development dependencies.
So now that everything is installed, it’s time to set things up.
Let’s start with karma.
Being a test runner that supports multiple frameworks, karma is reliant on you providing a number of configuration settings.
When karma starts up it tries to find a file called karma.conf.js in the current project folder.
You can create such a file by running the command karma init from the node_modules binaries folder.
Karma will run you though a few questions to try and set up a proper configuration file.
You can just press enter on all of these to go and see the end result.
Now although this creates a valid configuration file, this is nowhere near what karma needs to work with custom visual code.
Instead of typing a lot of boring configurations here, I’m instead going to paste a working configuration right into this file.
'use strict';
const recursivePathToTests = 'test/**/*.ts'
, srcRecursivePath = '.tmp/drop/visual.js'
, srcCssRecursivePath = '.tmp/drop/visual.css'
, srcOriginalRecursivePath = 'src/**/*.ts'
, coverageFolder = 'coverage';
module.exports = (config) => {
const browsers = [];
if (process.env.TRAVIS) {
browsers.push('ChromeTravisCI');
} else {
browsers.push('Chrome');
}
config.set({
browsers,
customLaunchers: {
ChromeTravisCI: {
base: 'Chrome',
flags: ['--no-sandbox']
}
},
colors: true,
frameworks: ['jasmine'],
reporters: [
'progress',
'coverage',
'karma-remap-istanbul'
],
singleRun: true,
files: [
srcCssRecursivePath,
srcRecursivePath,
'node_modules/lodash/lodash.min.js',
'node_modules/powerbi-visuals-utils-testutils/lib/index.js',
'node_modules/jquery/dist/jquery.min.js',
'node_modules/jasmine-jquery/lib/jasmine-jquery.js',
recursivePathToTests,
{
pattern: srcOriginalRecursivePath,
included: false,
served: true
}
],
preprocessors: {
[recursivePathToTests]: ['typescript'],
[srcRecursivePath]: ['sourcemap', 'coverage']
},
typescriptPreprocessor: {
options: {
sourceMap: false,
target: 'ES5',
removeComments: false,
concatenateOutput: false
}
},
coverageReporter: {
dir: coverageFolder,
reporters: [
{ type: 'html' },
{ type: 'lcov' }
]
},
remapIstanbulReporter: {
reports: {
lcovonly: coverageFolder + '/lcov.info',
html: coverageFolder,
'text-summary': null
}
}
});
};
This configuration tells karma to look into the proper folders for code and tests.
It also sets up links with jasmine, instanbul and the typescript compiler itself, so that everything runs neatly at the right time.
Instead of typing this yourself, I recommend you copy this file from the github repository and adjust it as needed for your visual.
Now another thing that karma needs to run is the Google Chrome browser.
If you haven’t installed this yet then pause the video now and go ahead and do it.
Another thing I need to do here is to tweak the typescript compiler configuration file.
"sourceRoot": "../../src/",
"declaration": true
The sourceRoot option just tells the compiler to use the original typescript files for source mapping, instead of the output.
This helps out with debugging in the browser as you’ll be able to debug against the original typescript code, instead of the compiled javascript.
Now the way the unit tests work is that they will run against the final build of your visual.
However, that final build is a javascript file, not a typescript one.
So to be able to make sense of that javascript file, you need a matching typescript declaration file and that’s what the declaration switch does there.
Enabling this makes the compiler produce a d.ts file with all the types and namespaces in your visual.
However, for that work, you will need to make a sacrifice.
And that is you will need to forfeit the ability to compile javascript files in your project.
I don’t know about you, but I don’t ever use JavaScript files in any new typescript projects, so I don’t mind this at all.
This won’t affect package dependencies as these are not compiled, they’re just included as-is in the final build.
Everything is now setup, but it’s important to understand what I did not setup in the first place.
And what I did not do here was to add any additional code files to this configuration file here.
As these new dependencies are for development only, I do not want them mingling with the final build output.
The same goes for the packaged external dependecies delcared in the pbiviz.json file.
Again, I don’t want to bring along any of these test related files to the build output.
The thing is, by not declaring these files, I’m going to have a problem.
When I’m writing my unit tests, Visual Studio will not recognize any of my test or visual code, as the test files and dependencies do not formally belong to the project.
The good news is, there is a way to compensate for that, so let me show you that while I create the first unit test.
So to hold all the tests, I’m going to create a folder here called test.
This, by the way, is the folder that I configured in the karma config file for tests.
If you want to store your tests somewhere else, by all means do so, but remember to update the karma configuration with your new location.
To compensate for the unit tests being outside of the project, I’m going to add a typescript file here and I’m going to name it references.
/// external libraries
///
///
///
///
/// power bi api and libraries
///
///
///
///
/// visual output
///
What this file does is to manually reference any project files that your test code needs.
This allows the unit tests to still have access to your whole code base, without being part of the typescript project.
Again, I recommend you copy paste this file from the Github repository and add your own references as you need.
The last item in the list may give an error the first time, with Visual Studio complaining it can’t find the file.
That’s because this is the new typescript declaration file that we just asked the typescript compiler to produce for the project.
You can force this file to show up, by running pbiviz package on the command line.
This will pick up the new settings and produce the file for you.
So with this out of the way, I’m going to create a new file called visual-spec.ts
In Jasmine lingo, test files are usually called specifications.
These are meant to hold a list of tests regarding a certain type, and if you’re doing test driven development, these test files will end up being the technical specification of that type.
Now this particular one is going to be a placeholder specification with a single test, just to make sure we have this working.
This test will only create an instance of the visual, and then verify that the visual was indeed created with no errors.
///
First, I need to reference that references file so I get almost everything I need here.
module powerbi.extensibility.visual.test {
describe("On the Visual class", () => {
// code here runs at the start of the test run
beforeEach(() => {
// code here runs before each test run
});
describe("the constructor method", () => {
it("must create a visual with no errors", () => {
// code here is the test itself
});
});
});
}
Then I’m going to paste here the skeleton for a jasmine specification.
So the way these Jasmine spec files work is that you can chain these describe functions to make them read like natural language.
Chaining these and other functions like before each allows you to avoid redundant test code by separating the many things that are usually repeated between tests, such as static mocked objects, test preparation code and test clean up code.
And as a bonus, Jasmine will take all the string you provide in the describe functions and concatenate them in a hierarchical manner to provide the final name for each test.
This means your tests are both readable on screen and on the final report as well.
If you’re curious to see this at work, keep watching.
This skeleton is the minimum set of Jasmine code you will usually start with.
The first describe function creates a test suite or context.
Directly inside the function I’m going to define some variables that will be shared across all the tests.
I’m also providing a description for this context.
This description is the start of an english language phrase.
That’s because I’m counting on this text being concatenated with other description, as defined in a sub context like this one.
I can call other Jasmine function here as well.
For example, the beforeEach function sets up a block of code that Jasmine will execute before running code on any other sub-contexts, which here, means this other describe function.
Now this chained describe function defines a sub-context.
This sub-context can have the same things as the parent, for example, I can add another beforeEach function here as well.
But instead, I want this this particular context to be used for all the tests related to a specific method, in this case the constructor.
The it function then defines a test.
This is where I’m going to write the actual test code.
Now the neat thing here is that the name of this test on the final report will be the hierarchical concatenation of all these descriptions that I’ve added.
So the name will be “On the Visual class the constructor method must create a visual with no errors.”.
You see.
The whole philosophy behind Jasmine is that of reducing redundancy to nothing and this applies even to the test names.
So let’s add some code.
/// specific imports
import Visual = powerbi.extensibility.visual.myLittleBarChartC2075CFB318240BC886AFAE5852EBCEB.Visual;
import IVisualHost = powerbi.extensibility.visual.IVisualHost;
import MockIVisualHost = powerbi.extensibility.utils.test.mocks.MockIVisualHost;
import MockIColorPalette = powerbi.extensibility.utils.test.mocks.MockIColorPalette;
import MockISelectionManager = powerbi.extensibility.utils.test.mocks.MockISelectionManager;
import MockITooltipService = powerbi.extensibility.utils.test.mocks.MockITooltipService;
import MockILocale = powerbi.extensibility.utils.test.mocks.MockILocale;
import MockIAllowInteractions = powerbi.extensibility.utils.test.mocks.MockIAllowInteractions;
First I have to import some types that I will need to write my tests.
Most of these are mocking helpers from the power bi testing utilities, which happens to be one of the dependencies I installed before.
One of these though, the top one, is a special one.
This is the Visual class as it exists in the final compiled file.
When compiling, pbiviz replaces the namespace of the visual classes with the GUID that you can see in the pbiviz.json file.
This is to ensure your code does not overlap with any other power bi code running on the host.
If you’re not using the GitHub project and you are using your own instead, then your namespace will be different.
As it is mine here in fact, as I copied this from my prep project for this course, so I need to fix too.
Now looking at this, these are types that I’m bound to use in different specs as I continue developing this visual.
Therefore, instead of having them here in this spec file, I may as well move them to the references file.
There you go.
Now let’s get back and start doing some testing.
// define global spec variables here
let host: IVisualHost;
let target: HTMLElement;
let palette: IColorPalette;
let selectionManager: ISelectionManager;
let tooltipService: ITooltipService;
let locale: MockILocale;
let allowInteraction1s: MockIAllowInteractions;
So as I want the first describe function to represent a test context for testing the entire visual, I’m going to define some variables here that I will keep using throughout all tests.
Note that I’m not instantiating anything here at all, I’m just definine the variable names.
That’s because I want these variables to be instantiated before every single test, so I know the tests are starting from a known state.
// mock constructor input variables
target = document.createElement("div");
palette = new MockIColorPalette();
selectionManager = new MockISelectionManager();
tooltipService = new MockITooltipService();
locale = new MockILocale({ "en": "en-US" });
allowInteractions = new MockIAllowInteractions(true);
host = new MockIVisualHost(palette, selectionManager, tooltipService, locale, allowInteractions);
Therefore I’m going to add the initialization code to the beforeEach() function.
This will ensure all these variables are reset when they need to.
By the way, note how I’m create the target html object by calling document.createElement.
You can do this in these tests because the code is in fact running in a browser window.
That’s the reason why karma requires a browser like Chrome.
It’s because you’re not supposed to create html objects outside a browser in the first place.
// create the visual for testing
let visual = new Visual({ element: target, host: host });
// ensure it exists
expect(visual).toBeDefined();
Now I just need to add the test code inside the it() function.
Here I just need to instantiate a new visual and state that I expect that visual variable to be defined.
Note how that reads well in English.
In fact, if you list all to() functions, you’ll find a whole bunch of stuff that you can use in your tests.
Anyway, that’s it really, pun intended.
That’s coding tests in Jasmine.
If you’re new to Jasmine, I do recommend you go and read the Jasmine documentation to learn more of the framework.
You can a find a link for that in the video description.
Okay, so this test is all written up.
I’m almost ready to run the test batch here.
Just one thing missing, that’s something that I really should have done a long time ago.
Let me go to package.json file…
"scripts": {
"postinstall": "pbiviz update 1.6.0",
"pbiviz": "pbiviz",
"start": "pbiviz start",
"package": "pbiviz package",
"lint": "node node_modules/tslint/bin/tslint \"+(src|test)/**/*.ts\"",
"pretest": "pbiviz package --resources --no-minify --no-pbiviz --no-plugin",
"test": "karma start"
}
And let me paste this in.
So what you see are npm custom commands.
These are scripts that you can run from the terminal using npm by mentioning the script name.
For example, if I want to issue a package of the project using npm syntax, I can now type npm run package.
Granted I did not save any typing by doing, but, what I did do, was to document terminal commands that would otherwise just live in my head.
When it comes to issuing a test run, there are actually two steps that need to happen.
First, you need to package the project using certain compilation options.
For example, you don’t want the final javascript code be minified and therefore unreadable.
Only then do you want to run the test batch, which requires another command.
By setting up these npm commands I can now enter npm run test and npm will run both the pretest script to package the code and the test script to run the test batch.
And that is what npm is doing right now, so let’s wait for it to finish.
And there you have it.
A successful test run.
Now, before we move on, let me show you what a failed test run looks like.
expect(false).toBeTruthy();
I’m gonna add some code that will force this test to fail.
And I’m gonna run this again.
Bang, it broke.
Let me scroll this stuff up so you can see something.
Look at that, right here.
“On the Visual class the constructor method must create a visual with no errors FAILED”
Those are all the descriptions concatenated to produce the name of the test.
Neat, isn’t it?
Let me just remove that code before moving on…
And let me run this again….
And there you go.
Now you can notice here that you have a coverage summary.
This is provided by the Istanbul package that comes with karma.
This is telling me that only about 25% of my code is covered by unit tests.
That’s nice to know, though what I would really like is to know what exact statements and functions are not yet covered by unit tests.
For that, we can look in a shiny new folder called coverage.
So here you can see this file called index.html.
When I open this with a browser, I can now see a report of source code folders and their stats regarding coverage.
And I can drill down to any given file and check what bits of code are not being covered at all.
For example, I haven’t written any test that touches the update function, so it’s being rightly flagged here.
Neat, isn’t it.
That’s it for testing and coverage.
There’s only one bit left and that’s linting.
Let’s go back to the project.
When I added this scripts block a while ago, I also added a command named lint.
This command is setup to run the tslint utility on your source and test files and look for any dodgy code.
So let’s see how that looks like.
Well, it doesn’t look like anything.
That’s because tslint needs a rules file, meaning a file that tells it what exact dodgyness to look out for.
This file is named tslint.json and needs to live in your project’s root folder.
{
"rules": {
"class-name": true,
"comment-format": [
true,
"check-space"
],
"indent": [
true,
"spaces"
],
"no-duplicate-variable": true,
"no-eval": true,
"no-internal-module": false,
"no-trailing-whitespace": true,
"no-unsafe-finally": true,
"no-var-keyword": true,
"one-line": [
true,
"check-open-brace",
"check-whitespace"
],
"quotemark": [
true,
"double"
],
"semicolon": [
true,
"always"
],
"triple-equals": [
true,
"allow-null-check"
],
"typedef-whitespace": [
true,
{
"call-signature": "nospace",
"index-signature": "nospace",
"parameter": "nospace",
"property-declaration": "nospace",
"variable-declaration": "nospace"
}
],
"variable-name": [
true,
"ban-keywords"
],
"whitespace": [
true,
"check-branch",
"check-decl",
"check-operator",
"check-separator",
"check-type"
],
"insecure-random": true,
"no-banned-terms": true,
"no-cookies": true,
"no-delete-expression": true,
"no-disable-auto-sanitization": true,
"no-document-domain": true,
"no-document-write": true,
"no-exec-script": true,
"no-function-constructor-with-string-args": true,
"no-http-string": [true, "http://www.example.com/?.*", "http://www.examples.com/?.*"],
"no-inner-html": true,
"no-octal-literal": true,
"no-reserved-keywords": true,
"no-string-based-set-immediate": true,
"no-string-based-set-interval": true,
"no-string-based-set-timeout": true,
"non-literal-require": true,
"possible-timing-attack": true,
"react-anchor-blank-noopener": true,
"react-iframe-missing-sandbox": true,
"react-no-dangerous-html": true
}
}
Now I don’t want to write all rules by hand here so instead I’m going to copy paste my rules from another project.
I recommend you do the same and grab this file from the GitHub repository for this series.
As always, you can find a link in the video description.
Now keep in mind that linting rules are really based on personal opinion of what typescript should look like.
There is no universal consensus on this, so my take is to go with something that helps your team write clean code without drowning into obsession.
So this is place now, let me try linting again…
And there you go, computer says no.
Let’s see why.
And look at that, it’s complaining about my dodgy code.
For example, resources.ts, line 9, missing semicolon.
Let’s see if this is true…
And it is indeed true.
Let me fix that.
And there you have it.
You have now setup a test runner, a test framework, a code coverage report and you even have some linting going.
You can finally call this a quality project.
That’s it for today.
And indeed that’s it for the bar chart.
This video concludes the main course on developing Power BI Custom Visuals.
If you have any questions, let me know and I’ll get back to you.
And if you’re new here and you want to be the first to know about new stuff as it comes, hit that subscribe button and enable notifications.
The next videos in this collection will be standalone and they will cover other types of charts, such as scatterplots, line charts, maps, etc, plus other specific issues when developing custom visuals.
Until then, take care and see you next time.