MythX and Continuous Integration (Part 1): CircleCI
By Dominik Muhs | Tuesday, January 28th, 2020
In the first post of a new series, we discuss integrating security considerations to a continuous integration pipeline, starting with CircleCI.
Continuous testing of applications can be hard to figure out. While it is difficult to measure CI/CD adoption, the blockchain ecosystem offers a great opportunity to adopt in-depth testing and continuous delivery pipelines where they make sense: right from the start.
At MythX we don’t mind which technologies you are using to get things done. Choosing the right tool for the right job means that service providers must not make assumptions about their client’s technologies. Security testing especially should be widely available and easy to use.
Behind the scenes the MythX team is working on the tech behind the smart contract analysis to make adding security to your workflow as simple as possible. For instance, we continuously run constructed samples against MythX in our QA process to keep our result quality as high as possible. This has several advantages:
- Avoid breaking changes. If things go wrong we will notice quickly and can iterate until the change is production-ready.
- Granular insights. Tests run on small change sets, one step at a time. This makes breaking changes obvious and allows for concise fixes.
- Save time on code reviews. We improve and review our internal test suites regularly. If they pass, confidence in the code’s correctness rises. That in turn keeps reviews focused on the business logic that matters.
There are plenty of reasons to use Continuous Integration (CI), but the above points have a nice side-effect: they also apply to smart contract development, or the software development lifecycle (SDLC) in general.
There are countless CI solutions out there and, as mentioned previously, we don’t mind which ones teams settle on.
That is why in this three-part series we will take a look at two of the most popular Continuous Integration solutions for open-source projects: CircleCI (this post), and Travis. Afterwards, we will generalize our approach to apply to other CI systems. Using the official MythX CLI it is fairly easy to get a basic test setup going.
Let’s jump right in!
CircleCI is configured using a YAML file located in
.circleci/config.yml to describe the build steps. It defines workflows, which in turn consist of jobs. Each of the defined jobs contain a number of steps that are run. (CircleCI offers great tutorials if you need a quick refresher.)
In our case, this will be downloading MythX CLI, and running it against a set of given smart contracts. First we define our basic config:
version: 2 jobs: build: docker: - image: circleci/python:3.8 steps: - checkout
This will define a job called
build. All the steps inside this job will be executed in a Docker container running Python 3.8. The MythX CLI also runs on Python 3.6+ and PyPy.
Now for the meaty stuff. Let’s open up a virtual environment and install the MythX CLI into it. This makes sure that our Python application does not collide with any other Python code running in CI, and it avoids a range of CircleCI permission issues.
In our first step we checkout the code at the commit the CI job was given.
- run: name: Setup virtual environment command: python -m virtualenv venv - run: name: Install MythX CLI command: | . venv/bin/activate pip install mythx-cli
There is nothing spectacular here. We define two steps, three commands, and we are set up. Now, submitting a contract to MythX can be done in a single command.
- run: name: Analyze with MythX command: | . venv/bin/activate mythx --ci --yes analyze --solc-version 0.5.0 contracts/
Well, actually two commands: Jump into the virtual environment, and then
analyze the code.
There are a few things in the
mythx call to note:
--citells the MythX CLI to return
1if a medium or high severity issue has been found.
--yesdisables the confirmation prompt asking the user whether they really want to submit X Solidity files.
analyzeis the subcommand that tells the CLI we want to send an analysis job to the MythX API.
--solc-versiondefines, you guessed it, the Solidity compiler version to compile the given files with.
contracts/is the directory the smart contracts live in. The MythX CLI will walk through this directory recursively and prepare analysis requests for each
.solfile that has been found.
By default the CLI submits scans in
quick mode. These return after roughly 120 seconds. Now our full CircleCI config looks like this:
version: 2 jobs: build: docker: - image: circleci/python:3.8 steps: - checkout - run: name: Setup virtual environment command: python -m virtualenv venv - run: name: Install MythX CLI command: | . venv/bin/activate pip install mythx-cli - run: name: Analyze with MythX command: | . venv/bin/activate mythx --ci --yes analyze --solc-version 0.5.0 contracts/
Below I have prepared a test repository with a vulnerable smart contract and connected it to CircleCI with the above configuration. As soon as the CI job is finished, the following output appears:
The CI check has failed and notified me of a vulnerability. Now I want to note exactly how to trigger it so I can write a test case and validate my fix.
So I have signed up for a free account on the MythX website and added my API token to the CI’s environment as
MYTHX_ACCESS_TOKEN. That way all my CI analyses are directly attached to my account and I can retrieve them later. In the dashboard’s analysis listing I find my submission:
Diving into it, I see the vulnerability overview that was displayed in my CI output again:
Scrolling down, I see that my code has been highlighted with the vulnerability locations:
For each vulnerability I also have the test case data at my fingertips.
With the click of a button I can copy the raw transaction data and easily bootstrap my Truffle test cases. This helps me to validate the fix in my code and avoid the issue from occurring in the future.
The MythX CLI is continuously improving (CI jokes, hah!) but I have written exhaustive documentation and a usage guide.
In addition to what I showed here, MythX CLI allows for more advanced patterns such as blacklisting vulnerabilities, opening custom analysis groups, delivering results in various output formats, and much more.
Do you want to store your reports as artifact files attached to the CI job? Just add the
-o parameter and configure the path in CircleCI. The possibilities are endless!
Feedback is a gift and a comment here or directly in the GitHub repository is greatly appreciated. In the next part, we will be talking about the same integration in Travis CI in part two. Stay tuned!
Have you signed up for a free MythX account yet? Get started today and integrate with your own CI pipeline.