Ceiba Web

Dipping Our Toes into ARK Core

Mario Vega

06 July 2018

Outline

Without a doubt, the best way I've found to understand code is to write it; reading it ranks as a close second. Accordingly, the best way to understand code for those who don't code is to read code, and that's what we'll focus on in this series.

For the first installment of Understanding Code, we'll be focusing on ARK Core v2. A recently released blockchain protocol written in JavaScript, ARK Core will provide an accessible jumping-off point into understanding blockchain for developers whose experience might be limited to web development (such as myself) as well as those who've only dabbled in programming or have no experience at all. I will assume very basic knowledge of programming concepts: if you can differentiate between an if statement and a for loop, you'll be fine.

There are two primary goals of the Understanding Code: ARK series. The first, as you might guess, is to understand how ARK Core works. We'll be working from the outside in: beginning with how ARK Core makes its information available to external developers, we'll explore deeper levels of the codebase until we understand the core mechanisms of the blockchain itself.

The second, broader goal is to impart technology enthusiasts of all levels of programming expertise with the necessary skillsets to begin understanding code on their own terms. We'll explore how to decipher a codebase you've never worked with before — how to understand developers' goals and the strategies they implement to achieve those goals.

The purpose of virtually all code is to solve a problem. Understand problems and their solutions and you've come that much closer to understanding code.

Before I begin, some necessary disclaimers. I chose ARK Core not because of some desire to pump my massive holdings: in fact, my newly-minted wallet address (AJAAfMJj1w6U5A3t6BGA7NYZsaVve6isMm) reveals that I have no ARK whatsoever. I selected ARK primarily at the suggestion of close friends, and also because of my previous familiarity with JavaScript through front-end development. I've discussed this column with precisely nobody who works for ARK in any sort of capacity — not even developers to ask their feedback on my analysis. So if you're looking for price predictions, trading advice, shilling or any related activities, I recommend looking elsewhere; you'll find nothing here but code and words explaining code.

Getting Started — Where To Look

As you probably already know if you've read this far, the best way of exploring open-source code is through exploring a project's GitHub page. You can find the code for ARK Core v2 here.

Trying to navigate at GitHub as a non-coder can be pretty intimidating. Let's break it down.

I've created a video for this column that goes through the basics of GitHub using Ark Core v2 as the base project. In another column, I'll go through GitHub in detail, including its relation to Git, the program that powers GitHub's functionality and allows developer to release code in versions. Until then, here's the video breakdown:

<iframe width="560" height="315" src="https://www.youtube-nocookie.com/embed/quyLwJzw8cA" frameborder="0" allow="autoplay; encrypted-media" allowfullscreen></iframe>

I'll assume some basic knowledge of GitHub, either through your own experience or through watching my totally-awesome-and-not-improvised-at-all video above. With no further ado, the code!

This is what the Ark Core v2 repo looks like at first glance. Lots of files with periods in front of their names, and a couple folders.

The important thing to understand here is that the "core" repo is simply a central place to organize all of the packages that comprise Ark Core. You can find all of the individual packages within the packages directory, and we'll dive much more into each individual package later in this column as well as later in this series. Suffice it to say for now that most of the files at this top level involve configuration options to make the process of developing all of these packages easier.

With any JavaScript project, the first place to look to start understanding how the different pieces fit together is the package.json file. This file is what JavaScript uses to figure out which external libraries need to be installed for this library to work. A feature of virtually all modern programming languages is the ability to leverage code written by third parties to avoid having to reinvent the wheel every time your project requires new functionality. By inspecting the package.json, we can piece together what functions a given library is leveraging from the broader JavaScript community — an important clue in understanding the purpose and architecture of the codebase. Accordingly, we'll be looking at lots of package.jsons in our quest to understand the various libraries that make up Ark Core.

Here is a screenshot of the top-level package.json, as taken from my computer:

First off, you should note that this file, like all other files ending in .json, is formatted in a ubiquitous file format called, perhaps unsurprisingly, JSON. Short for JavaScript Object Notation, JSON is used to present information in a way that's equally accessible to humans and computers. As easy as it is to look at this file and understand the information hierarchy (and if it's not easy, it will be), it's equally easy for the vast majority of programming languages to parse this document and take action accordingly.

The two keys we're most interested in here are devDependencies and scripts. The devDependencies key refers to a list of packages that this library depends on, as well as what versions of those packages the library requires. For example, we know from reading this document that this library requires lerna to function properly, and that the version of lerna must be at least 2.11.0. The scripts key is a list of commands that can be run on this package. In this case, there are commands like test, which runs tests across the entire Ark Core, commit, which commits code to GitHub using commitzen, and more.

As mentioned previously, most of the libraries we'll find in this top-level package.json help with managing all of the Ark Core packages efficiently. You can find out more information about any of these libraries by searching the central JavaScript package library (NPM, or Node Package Manager) at https://www.npmjs.com/. Briefly, I'll go over the most important ones here:

Notable Packages

Lerna

Probably the most relevant package here is Lerna. In addition to having a badass logo, Lerna is the industry-standard package for packages that have multiple packages. All "Yo dawg, I heard you like packages..." jokes aside, Lerna is an absolute must-have when building JavaScript libraries at the scale of Ark Core, as it makes managing dependencies across the different projects easier. From looking at the devDependencies key, you can see that every dependency has its own version — Lerna helps keep those dependencies consistent across your project to coordinate work across all packages. You'll notice that many commands in the scripts section refer to Lerna, such as bootstrap, clean and others.

Eslint

You'll notice lots of references to Eslint. This package is important for two primary reasons: maintaining a consistent coding style, and preventing some classes of bugs from making their way into code. When you have many different developers working on a single project with their own coding styles, the resulting effort can feel a bit disjointed, like a book with multiple authors who don't talk to each other before publishing. Eslint fixes those inconsistencies by applying a single coding style to every package in the repository.

In addition, unlike some other programming languages, JavaScript is dynamically typed, meaning that programmers don't have to declare a type for each variable they create. While this allows for faster and more agile programming, dynamic types also introduce the possibility that your program will try to interact with a variable in a way that its type does not allow.

Without Eslint, you wouldn't discover that error until your code is actually run. For example, if you defined two variables as "apple" and "banana", and you then tried to divide "apple" by "banana" within your code, JavaScript wouldn't catch that error until you (or worse, your application's users) ran the program and encountered it. Eslint solves both of these problems elegantly and flexibly using customizable standards determined by each project, making it a must-use in a large program like Ark Core.

Jest

Jest, and the closely-related Jest-Extended, are testing libraries. We'll cover tests in short order, but the SparkNotes is that tests are often the quickest and easiest way to understand how a library operates. That's because, when designed well, tests are describe how your code functions at the highest level possible. The how of your code is hidden, but the what of your code is easily readable and accessible.

Putting It Together

From this description, it immediately becomes clearer what some of the mysterious "dot files" are doing for this repository. lerna.json contains customizations that ARK Core has specified for Lerna to use: where to find the packages it should bundle within the repository, for example. .eslint specifies how eslint should lint the repository, and .eslintignore tells eslint which files it should ignore while doing so. Similarly, jest.config.js provides Jest with information that allows it to run tests within the specific context of ARK Core: which tests to always run, which tests should only run occasionally, where it should find tests to run, etc.

Importantly, there's nothing in the top-level folder that refers to anything blockchain-related. To keep things organized, all code that actually powers the ARK blockchain is contained within the packages folder. The docker folder is mostly important to developers looking to replicate the ARK toolchain on their own systems, and we'll ignore it for the purposes of this tutorial. With the plugins folder being empty, let's inspect the packages folder and get to the meat and potatoes.

Previewing the Packages

Whoa. That's a lot of folders. And if we poke around a bit, we see that each package has a handful of their own files and folders, and even their own package.json. What do we do now? Do we give up? Do we jump straight into the core-blockchain folder and see if we can shortcut our way to blockchain enlightenment?

Not quite! We'll dive a little bit into our first package to start the next column in this series, but keep in mind our objective here is to work from the outside in. With that in mind, let's take another look at the ARK Core Readme to figure out what the best entry point is to experience the blockchain from a public perspective before delving into the blockchain's internals.

This is a screenshot from the relevant section in the Readme. At a glance, there are two packages that seem to fit our bill of requirements: client and core-api. Both of these seem to deal with how the public interacts with the blockchain — client is a client-side library that anybody can integrate into their own website, and core-api describes how an ARK node should respond to requests from clients.

As mentioned before, looking at a package's tests is a good way of preview what its functionality is. With that, let's take a brief look at each packages' tests to see which might be better to start with.

Client Tests

Core-API Tests


Above, I've pulled out some of the first tests from each folder. We'll go more into these tests, as well as how to find them and understand them, in the next column. But at a glance we can see that the client package tests lots of non-blockchain-related functionality: how to properly send an HTTP request, how to make sure URL typos won't break the package, etc. These are important concepts, no doubt, but understanding them won't get us much closer to understanding how ARK works. By contrast, the core-api gets right into the details of the blockchain itself. We can see that this test expects to retrieve a list of all blocks. If we figure out what code this test is designed to cover — in other words, what code needs to be working properly for this test to pass — we'll know how the API retrieves blocks from the blockchain. From there, our journey into understanding how those blocks are created, stored and retrieved can begin in earnest.

So, we'll be looking at core-api in the next column! If space permits, perhaps we'll cover another package as well, but let's not get too ambitious. It's more important that we get it right than get it quickly.