CircleCI Blog

This is the second of a two-part blog post on detecting and correcting errors with Rollbar and CircleCI, by guest contributor Jason Skowronski. Read part I here.

When you’re practicing continuous delivery, it’s essential that you monitor your application so you know that it’s performing well after deployments. You need to be notified immediately if something is wrong or if users are having a poor experience so you can resolve the issue quickly. When your monitoring solution can also tell you which code changes caused the error, you can save valuable time troubleshooting.

Today’s blog post is the first of a two-part series written by Jason Skowronski, who serves as the lead editor for technical content at Rollbar. Jason began his career as a software developer at Amazon and now enjoys being a developer advocate for the latest technologies.

Application errors can cause frustrating problems for customers that may ultimately lead to losing their trust and business. Experienced developers know what it’s like to have a critical production problem and spend minutes or perhaps even hours diagnosing a tricky problem. It’s even harder to diagnose when several developers are making changes and deployments in parallel.

Rollbar is an error monitoring solution that can tell you what errors occurred after a deployment and show you the deployment and code change that likely caused them. It integrates with your continuous integration and delivery (CI/CD) system to track when deployments are promoted to production. When an new error occurs, it looks up the deployed version in your source code repository like GitHub to identify what code was changed and who changed it. This will help you narrow down errors due to code bugs faster.

This is a guest post by Pavlos-Petros Tournaris. It originally appeared on his blog here. Pavlos-Petros works as an Android software engineer at Workable. We hope you enjoy!

Nearly a year and a half ago, Facebook released Litho as an open source project. Litho is an Android framework, which lets you define your UI in a declarative way. It immediately got my interest and I started getting my hands dirty with some examples and pet projects.

Static Site Generators (SSGs) such as Hugo and Jekyll are all the rage nowadays. These static on the backend and JavaScript on the frontend websites are called JAMStacks. Typically, we test them in two ways:

• via the SSG successfully building the site
• and HTMLProofer

What if we wanted to do more? Let’s walk through a new tool I made for testing markdown files and how it improved the accuracy of CircleCI docs examples.

Today we made a small change to the navigation inside CircleCI. Where you saw the word “builds” you will now generally see the word “jobs”. This change has zero impact on the functionality of CircleCI; all your projects should behave identically to yesterday.

That’s all you need to know to enable you to keep using CircleCI just as before. If you’re curious about why we’ve made this change, read on.

Part I: Introduction to package management in CI/CD

Overview

In this post, we’re going to explain some fundamental concepts around software packages and package management and how to combine them with CI/CD to build a software delivery pipeline.

We’ll go over software packages and repositories, continuous delivery, and automating the building, testing and release stages of the software development process.

possibly updated info here? https://discuss.circleci.com/t/useful-tips-and-best-practices-when-migrating-to-circleci-2-0/23075

CircleCI specific

• Commands listed in the steps can only be run in the first container listed in the docker section.
• Run builds frequently to test the config thus far. If something breaks, you know what changed since the last build.
• Don’t add workflows initially; wait until you have a functional build.
• Build a config manually from scratch, but use the config-translation endpoint as a reference
• You can’t define env vars with env vars in the environment section of the config
  • The workaround is to echo the vars into $BASH_ENV
    • This only works with bash, not sh (Alpine images only have sh)
• Conditionally run commands with bash if statements
  • if [ $CIRCLE_BRANCH = “master” ] ; then ./ci.sh ; fi
• Conditionally halt the build at that step with circleci step halt
  • Allows you to use setup_remote_docker conditionally by halting
• The Timezone can be changed just by defining an env var
  • TZ: “/usr/share/zoneinfo/America/New_York”
• Running the build out of /dev/shm (ie., /dev/shm/project) can speed up certain projects
  • Some things like Ruby gems can’t be loaded out of shared memory. They can be intalled elsewhere in the system (~/vendor) and symlinked in.
• Instead of prefixing a lot of commands with sudo, consider setting the shell to sudo for that run
  • shell: sudo bash -eo pipefail
• Docker builds and docker-compose should generally be run in the machine unless language-specific tools (Ruby, Node, PHP, etc.) are required beforehand, then the remote environment is sufficient.
• Some tasks can be set to run in the background to save overall build time, but be careful of running out of resources
• Different resource_class sizes can be beneficial and it’s worth trying them to see their impact. It could have no impact at all.
• The $PATH can be set to a string. If you don’t know your Docker image’s $PATH, just run it and echo $PATH or take a look at the output of env.
• The sha of an image can be referenced under the Spin up Environment step. An image can be hardcoded to that sha value to make it immutable.
• Service containers can be run with custom flags
  • command: [mysqld, –character-set-server=utf8mb4, –collation-server=utf8mb4_unicode_ci, –init-connect=’SET NAMES utf8mb4;’]

Onboarding from 1 -> 2 specific

• Note that $CIRCLE_ARTIFACTS and $CIRCLE_TEST_REPORTS are not defined in 2.0
  • You can define them yourself, but be sure to mkdir -p $CIRCLE_ARTIFACTS $CIRCLE_TEST_REPORTS if you do.
• Migrating Linux & macOS in one repository (like React Native) should involve opening one branch for each Linux & macOS before combining the two configs into one workflow.

• You can’t sudo echo - pipe it like this: echo “192.168.44.44 git.example.com”

  sudo tee -a /etc/hosts`

• Fonts are different between Ubuntu and Debian systems.
• Apache 2.2 and 2.4 configs are fairly different - make sure to upgrade your 2.2 configs.
• Don’t forget all the commands inferred automatically by 1.0 and commands manually stored in the UI

Language specific

Python

• Typically expects classnames and not filenames to run tests

Ruby

• Ruby files can load in a different order than expected on AUFS
• Define $RAILS_ENV and $RACK_ENV as test (this was automatic in 1.0)

Java(-based)

• Java (apps, tools, and services) will OOM (run out of memory) because it doesn’t recognize how much RAM is available. An environment variable should be defined. If it’s still OOMing, a bigger container is necessary.
  • https://circleci.com/blog/how-to-handle-java-oom-errors/
• Scala projects can have filenames that are too long, include the -Xmax-classfile-name flag.
  • https://discuss.circleci.com/t/scala-sbt-assembly-does-not-work/10499/10 scalacOptions ++= Seq( “-encoding”, “utf-8”, “-target:jvm-1.8”, “-deprecation”, “-unchecked”, “-Xlint”, “-feature”, “-Xmax-classfile-name”, “242” <= add here ),

Browser testing

• Tests can sometimes be flaky and fail for seemingly no reason. Some people opt to re-run their failing browser tests automatically. The drawback is corrupting the timing data.
• Take screenshots of failed tests to make debugging easier
• VNC can be installed & used. The browser can be dragged around in VNC after installing metacity. Run this from one of our browsers images: ssh -p PORT ubuntu@IP_ADDRESS -L 5902:localhost:5901 # To connect via SSH sudo apt install vnc4server metacity vnc4server -geometry 1280x1024 -depth 24 export DISPLAY=:1.0 metacity & firefox &

Docker specific

• Build a Docker image on a cron job
  • Build weekly, daily, or whenever you need
    • Possible to trigger a new Docker image build via the API easily
  • Include dependencies like node_modules in the image
    • Helps mitigate issues from a DNS outage
    • Keeps the dependencies version controlled
    • Even if a module disappears from node’s repos, the necessary depenencies to run the application are safe.
  • A private image can include private gems and private source cache
• There is no socket file for databases so the host variables need to be defined ($PGHOST, $MYSQL_HOST) to specify 127.0.0.1, forcing TCP
• Using CircleCI convenience or official Docker Hub images increases the chance of having your image cached on the host
  • Building off these images will reduce the number of image layers that need to be downloaded
• Using the -ram variation of a container will run the given daemon in /dev/shm
• Building custom images with everything pre-installed speeds up the build and adds reliability
  • Heroku (as an example) could push a bad update to their installer, breaking your builds
• The dockerize utility can be used to wait for service containers to be available before running tests
  • https://github.com/jwilder/dockerize
• ElasticSearch has their own Docker registry from which to pull
  • https://www.docker.elastic.co/
• Containers can have names so multiple of a given service can run on the same port with different hostnames
• Privileged containers can be run in the remote environment and the machine
• Volumes can’t be mounted from the base Docker executor into the remote environment
  • docker cp can transfer files
  • Volumes referenced will be mounted from the within the remote environment into the container

Fun facts:

• You are limited by your imagination in CircleCI 2.0
• The shell can be set to Python to just execute arbitrary Python in the YAML - run: shell: /usr/bin/python3 command: import sys print(sys.version)
• You can be clever with bash to achieve whatever you need for i in {1..5}; do curl -v $ENDPOINT_URL && break || sleep 10; done

英文のあとに日本語訳が続きます。

Today we’re proud to announce the formation of CircleCI G.K., a subsidiary of CircleCI. While we have worked with Japanese developers for some time, this is our first official international office, and a sign of our commitment to the Japanese market.

CircleCI in Japan

Japan has long been a focus for CircleCI: it’s one of our largest markets, and we work with some incredibly innovative companies there, like CyberAgent, Mercari, DMM, DeNA, CrowdWorks, and many others. Establishing a dedicated presence in Tokyo will help us better serve those customers and make the experience of using CircleCI better for all of our users in Japan.

Last week, we partnered with Code2040 to host their first PopUp event here at CircleCI HQ. The PopUp series is designed to engage the broader community, in particular rising Black and Latinx tech professionals. We were delighted to host the Code2040 team, an incredible panel of guest speakers (including our very own Jose Browne!) and an audience of 75 attendees.

Having security controls in place is no longer the only consideration when assessing one’s risk factors. Even when security controls are implemented, they’re often not appropriately permissioned. For example, it’s not uncommon for critical service-level accounts to be given excessive system privileges. This increases attack vectors within a system should that account be compromised. This compromised account could then execute malicious code or access very sensitive data that could present any number of issues for an organization. One way to prevent these types of security incidents is to implement the concept of Principle of Least Privilege.