Part 3: MythX ❤️ Continuous Integration (DIY)

By Dominik Muhs | Monday, March 16th, 2020

This is the third and last post in the MythX series on integrating security analysis of smart contracts into your Continuous Integration (CI) system. In the first part we built a CircleCI configuration. In the second part we built a small and beautiful Travis CI configuration.

“But I am using a completely different CI system!” – You, maybe.

Fret not. The avid reader might have noticed a common pattern in the past two posts: In setups we used the MythX CLI and a basic Python runtime. If you have not read the previous articles yet, it is recommended to do so. Do not worry, I will wait here.

Generalizing our Approach

Sending things to MythX for analysis is rather simple if you are using one of our many tools and integrations. Roughly speaking we always do the following:

  1. Download and set up an official tool
  2. Run the tool on one or multiple target files
  3. Wait for the results be returned by the MythX API
  4. If the report contains anything critical, fail the job

Available Tools

The MythX analysis engine is made available through a REST API. This means that there are analysis tools readily available for most common languages and frameworks. Are you using Truffle? Check out the MythX Truffle plugin. Do you prefer Python and use Brownie? Brownie ships with built-in MythX integration – no setup required. Or do you prefer a minimal JS client to eliminate the Python dependency? Thy sabre! Or maybe you want it as comfortable as possible and not configure your CI server at all. Then Guardrails is for you!

Or simply roll your own. The MythX documentation is a good starting point to build the tool that perfectly fits your needs. You could also dive head-first into our OpenAPI documentation if you prefer looking at data structures instead of text blocks. Whatever your choice is, the MythX team welcomes and supports all builders!

Advanced Usage

This blog series is meant to be a starting point for CI integrations. It is by no means exhaustive. As the main developer behind the MythX CLI I can’t help it however to throw some cool ideas for advanced users out there. All setups described below are possible with the current feature set of the MythX CLI. If you have implemented them in your project, please share your experiences with me. I am determined to polish the application even more, make it more reliable, and I would love to feature your use case in another blog post!

Long Analysis Jobs

So far we have only discussed “quick” mode submissions. These usually take about 120s to finish. A CI server can wait this long. But what if you have a MythX subscription that allows you to do “standard” or “deep” mode scans? These allow for up to 90 minutes of scan time.

Given these long analysis times the CI job would probably time out and fail in any case. The MythX CLI solves this problem by introducing an --async flag. If this flag is given, a list of job IDs will be printed and the CLI will not wait for the analyses to be finished.

This in turn means that scan results are not available inside the CI logs. The submitted scans can be viewed in the MythX dashboard once they are finished. Running asynchronous analyses is as simple as:

mythx analyze --async 

Storing Artifacts

Sometimes you want to persist files from your CI job. Let’s say you have a dedicated security verification pipeline where people are continuously monitoring the state of your smart contracts and fix vulnerabilities once they come up. You now want to provide this pipeline with data as soon as the MythX analysis has finished. Disregarding the specifics of your CI system, the MythX CLI allows for file outputs in various formats. E.g. we can take the analysis results, format them in pretty-printed JSON and store them in a file:

mythx --output security.json --format json-pretty analyze 

In your CI server you can now specify the output directory as an artifact file and e.g. directly push it to AWS S3 with Travis.

Filtering Results

There might be things you do not want to see in the MythX CLI output. For that reason, we have added two features: Severity filtering and SWC blacklisting. If you are tired of us reminding you that your code has a floating pragma statement, simply blacklist this SWC.

mythx analyze --swc-blacklist SWC-103 

If this is not enough, simply add more SWC IDs as a comma-separated list to the parameter. But maybe low severity issues in general are none of your concern and you are only interested in everything above Medium. That’s what the severity filter was made for:

mythx analyze --min-severity high 

Conclusion

Security checks in continuous integration systems are not too hard. When the tooling is right and a little setup time is invested, it is not impossible to always have updated security reports at your fingertips. The Ethereum ecosystem is growing rapidly with innovations in smart contract development being made on a daily basis. It fills me with joy to see such a huge amount of projects that try to make a difference, often just driven by curiosity and passion.

At the same time, it fills me with uncertainty how we secure such massive growth. In a recent analysis with our friends over at Alethio we have found out that the number of smart contract vulnerabilities is on a downward trend. Automated tools, dedicated security auditors, and passionate developers start to value security more and more. I hope that this blog series, and MythX in general can further contribute to this trend.

Stay safe out there!

Share this post:

MythX and Continuous Integration (Part 2): Travis

By Dominik Muhs | Tuesday, February 4th, 2020

In the second part of this series on continuous integration, we will build an easy first integration of the MythX API into the Travis continuous integration platform.

Other posts in this series:

This three-part series is about integrating MythX into Continuous Integration systems. In the first part of this series I have shown how to automatically check for smart contract vulnerabilities in CircleCI. In this part we will take a look at another popular open-source CI system: Travis.

Travis CI

Similar to CircleCI, Travis is configured using a YAML file. This time it is located in the project’s root directory as .travis.yml. The basic concepts of Travis are slightly different from other solutions, however. This greatly simplifies build definitions and job descriptions. This will work in our advantage here.

The tl;dr is that Travis has stages. Each stage consists of multiple jobs that are executed in parallel. These jobs can be grouped into builds. One of these builds will be a collection of jobs that do the MythX security analysis steps.

Sounds complicated? Once we dive into the actual configuration it will turn out to be really easy. But do not trust me, read on!

The basic setup

We will again use Python to run the MythX CLI for all our analysis needs. In the previous post we had to use a virtual environment to isolate our dependencies. In Travis we will rely on a dedicated stage that runs in parallel to all the other things we want to do. So we don’t have to isolate our dependencies. Absolute freedom!

Let’s start writing our configuration file and specify the Python version to use:

language: python
python:
  - 3.7

Before the magic starts we need to install the MythX CLI. We do this in the install phase – before any further steps are executed:

install:
- pip3 install mythx-cli

Spoiler: This line is also how you would install the MythX CLI on your machine as well (with an optional --user added into the mix.)

Submitting to MythX

Now for the analysis magic of submitting our smart contracts to MythX and making the build fail in case anything bad is found:

script:
  - mythx --ci --yes analyze --solc-version 0.5.0 contracts/

And that’s it. To dissect the mythx command a bit further:

  • --ci tells the MythX CLI to return 1 if a medium or high severity issue has been found.
  • --yes disables the confirmation prompt asking the user whether they really want to submit the Solidity files.
  • analyze is the subcommand that tells the CLI we want to send an analysis job to the MythX API.
  • --solc-version defines (you guessed it) the Solidity compiler version to compile the given files with.
  • contracts/ is the directory the smart contracts live in. The MythX CLI will walk through this directory recursively and prepare analysis requests for each .sol file that has been found.

By default the CLI submits scans in quick mode. These return after roughly 120 seconds. So our complete Travis CI configuration file is short and sweet:

language: python
python:
  - 3.7

install:
- pip3 install mythx-cli

script:
  - mythx --ci --yes analyze --solc-version 0.5.0 contracts/

Run it!

I have prepared a test repository with a vulnerable smart contract and connected it to Travis with the above configuration. As soon as the CI job is finished, the following output appears:

The CI check has failed and notified me of a vulnerability. Now I want to know exactly how to trigger it so I can write a test case and validate my fix.

The benefit of signing up for an account on the MythX website (adding my API token to the CI’s environment as MYTHX_ACCESS_TOKEN) is that all my CI analyses are directly attached to my account and I can retrieve them later.

In the MythX dashboard I find my submission:

Diving into it, I see the vulnerability overview that was displayed in my CI output again:

Scrolling down, I see that my code has been highlighted with the vulnerability locations:

For each vulnerability I also have the test case data at my fingertips. Nice!

With the click of a button I can copy the raw transaction data and easily bootstrap my Truffle test cases. This helps me to validate the fix in my code and avoid the issue from occurring in the future.

What’s next?

The MythX CLI is continuously improving (CI jokes, hah!). I have written exhaustive documentation and a usage guide, which allows for more advanced patterns such as blacklisting vulnerabilities, opening custom analysis groups, delivering results in various output formats, and much more.

Do you want to store your reports as artifact files attached to the CI job? Just add the -o parameter and configure the path in Travis.

As always, feedback is a gift, and a comment here or directly in the GitHub repository would be greatly appreciated.

Next up, we will generalize our CI approach in part three. Stay tuned!

Share this post:

MythX and Continuous Integration (Part 1): CircleCI

By Dominik Muhs | Tuesday, January 28th, 2020

In the first post of a new series, we discuss integrating security considerations to a continuous integration pipeline, starting with CircleCI.

Conveyor belts. Because continuity. Get it?

Continuous testing of applications can be hard to figure out. While it is difficult to measure CI/CD adoption, the blockchain ecosystem offers a great opportunity to adopt in-depth testing and continuous delivery pipelines where they make sense: right from the start.

At MythX we don’t mind which technologies you are using to get things done. Choosing the right tool for the right job means that service providers must not make assumptions about their client’s technologies. Security testing especially should be widely available and easy to use.

Behind the scenes the MythX team is working on the tech behind the smart contract analysis to make adding security to your workflow as simple as possible. For instance, we continuously run constructed samples against MythX in our QA process to keep our result quality as high as possible. This has several advantages:

  • Avoid breaking changes. If things go wrong we will notice quickly and can iterate until the change is production-ready.
  • Granular insights. Tests run on small change sets, one step at a time. This makes breaking changes obvious and allows for concise fixes.
  • Save time on code reviews. We improve and review our internal test suites regularly. If they pass, confidence in the code’s correctness rises. That in turn keeps reviews focused on the business logic that matters.

Why CI?

There are plenty of reasons to use Continuous Integration (CI), but the above points have a nice side-effect: they also apply to smart contract development, or the software development lifecycle (SDLC) in general.

There are countless CI solutions out there and, as mentioned previously, we don’t mind which ones teams settle on.

That is why in this three-part series we will take a look at two of the most popular Continuous Integration solutions for open-source projects: CircleCI (this post), and Travis. Afterwards, we will generalize our approach to apply to other CI systems. Using the official MythX CLI it is fairly easy to get a basic test setup going.

Let’s jump right in!

CircleCI

CircleCI is configured using a YAML file located in .circleci/config.yml to describe the build steps. It defines workflows, which in turn consist of jobs. Each of the defined jobs contain a number of steps that are run. (CircleCI offers great tutorials if you need a quick refresher.)

In our case, this will be downloading MythX CLI, and running it against a set of given smart contracts. First we define our basic config:

version: 2
jobs:
  build:
    docker:
      - image: circleci/python:3.8
    steps:
      - checkout

This will define a job called build. All the steps inside this job will be executed in a Docker container running Python 3.8. The MythX CLI also runs on Python 3.6+ and PyPy.

Now for the meaty stuff. Let’s open up a virtual environment and install the MythX CLI into it. This makes sure that our Python application does not collide with any other Python code running in CI, and it avoids a range of CircleCI permission issues.

In our first step we checkout the code at the commit the CI job was given.

- run:
  name: Setup virtual environment
  command: python -m virtualenv venv
- run:
  name: Install MythX CLI
  command: |
    . venv/bin/activate
    pip install mythx-cli

There is nothing spectacular here. We define two steps, three commands, and we are set up. Now, submitting a contract to MythX can be done in a single command.

- run:
  name: Analyze with MythX
  command: |
    . venv/bin/activate
    mythx --ci --yes analyze --solc-version 0.5.0 contracts/

Well, actually two commands: Jump into the virtual environment, and then analyze the code.

There are a few things in the mythx call to note:

  • --ci tells the MythX CLI to return 1 if a medium or high severity issue has been found.
  • --yes disables the confirmation prompt asking the user whether they really want to submit X Solidity files.
  • analyze is the subcommand that tells the CLI we want to send an analysis job to the MythX API.
  • --solc-version defines, you guessed it, the Solidity compiler version to compile the given files with.
  • contracts/ is the directory the smart contracts live in. The MythX CLI will walk through this directory recursively and prepare analysis requests for each .sol file that has been found.

By default the CLI submits scans in quick mode. These return after roughly 120 seconds. Now our full CircleCI config looks like this:

version: 2
jobs:
  build:
    docker:
      - image: circleci/python:3.8
    steps:
      - checkout
      - run:
          name: Setup virtual environment
          command: python -m virtualenv venv
      - run:
          name: Install MythX CLI
          command: |
            . venv/bin/activate
            pip install mythx-cli
      - run:
          name: Analyze with MythX
          command: |
            . venv/bin/activate
            mythx --ci --yes analyze --solc-version 0.5.0 contracts/

Run it!

Below I have prepared a test repository with a vulnerable smart contract and connected it to CircleCI with the above configuration. As soon as the CI job is finished, the following output appears:

The CI check has failed and notified me of a vulnerability. Now I want to note exactly how to trigger it so I can write a test case and validate my fix.

So I have signed up for a free account on the MythX website and added my API token to the CI’s environment as MYTHX_ACCESS_TOKEN. That way all my CI analyses are directly attached to my account and I can retrieve them later. In the dashboard’s analysis listing I find my submission:

Diving into it, I see the vulnerability overview that was displayed in my CI output again:

Scrolling down, I see that my code has been highlighted with the vulnerability locations:

For each vulnerability I also have the test case data at my fingertips.

Nice!

With the click of a button I can copy the raw transaction data and easily bootstrap my Truffle test cases. This helps me to validate the fix in my code and avoid the issue from occurring in the future.

What’s next?

The MythX CLI is continuously improving (CI jokes, hah!) but I have written exhaustive documentation and a usage guide.

In addition to what I showed here, MythX CLI allows for more advanced patterns such as blacklisting vulnerabilities, opening custom analysis groups, delivering results in various output formats, and much more.

Do you want to store your reports as artifact files attached to the CI job? Just add the -o parameter and configure the path in CircleCI. The possibilities are endless!

Feedback is a gift and a comment here or directly in the GitHub repository is greatly appreciated. In the next part, we will be talking about the same integration in Travis CI in part two. Stay tuned!

Have you signed up for a free MythX account yet? Get started today and integrate with your own CI pipeline.

Share this post:

MythX Pro Security Analysis Explained

By Bernhard Mueller | Tuesday, November 19th, 2019

MythX recently went live with a new Pro upgrade that offers more powerful analysis features than the free version. In this article I’ll explain how the new “full” analysis mode affects the performance of MythX.

MythX is a smart contract security service that integrates multiple analysis techniques. The MythX Pro plan comes with a new analysis mode called Full mode. In this mode, submitted contracts are subjected to a thorough fuzzing campaign and deep inspection using symbolic analysis. It discovers complex security issues and can be used for checking the correctness of smart contracts.

Background

When you submit a smart contract to MythX, the analysis service spawns a number of workers that perform various analysis tasks in parallel. Each worker is given a maximum time budget it may spend on testing the code. The more computing time is available, the higher the coverage achieved by the analysis engine (at the cost of having to wait longer for the results).

Continue…
Share this post: