Create CLIs For 2023 Using Deno

Introduction

We are in 2023 and my world still evolves around CLIs¹. I enjoy looking at my terminal and piping commands together. Knowing how to do stuff with CLIs is empowering and gives the impression of really understanding how things are built under the hood.

At work, I like to create a CLI where I put all my utilities bundled in a single executable. Usually, the CLI starts as a bunch of aliases and evolves into a complex tool that is used by many people.

In this post, I will detail why my next CLI will be written in deno.

Requirement for a CLI in 2023

User Requirements

It is 2023 and expectations for CLIs should be higher than in 1973 or 2013. The experience has shown that good CLIs should:

1. Have a nice, backward-compatible, user interface. Users now expect a “git like” command user interface with commands, sub-commands and descriptive help messages.

2. Be easy to install. The typical old librarian should be able to install a good CLI.

3. Be upgradable. A good CLI should allow its users to upgrade it.

4. Be stateful. A good CLI should allow its users to set personal properties such as credentials and use them as default in subsequent executions.

Technical Requirements

To fulfill the user requirements, it is wise to add technical requirements to the selected stack that will ease as much as possible the implementation of a proper CLI.

1. The selected programming language should have a mature framework to build a CLI.

2. The programming language should be part of a large ecosystem. The CLI programmer does not want to build a library as much as glue together libraries and services.

3. The project should not be over complicated to build. Minimal CI/CD² should be required to maintain the project.

Security Requirements

In 2023, a user should not give all rights to software. Software should be sandboxed and ask the user for permissions explicitly.

Writing a CLI in Deno.

Without being a tutorial, this section will detail the steps required to write a CLI in deno.

The Solution

The project will be written in typescript using deno as the runtime. cliffy will be used as the CLI framework. The state will be persisted in a SQLite database.

Deno is a javascript runtime environment that has its own ecosystem and can use libraries published on npm. Cliffy is a very capable CLI framework with good documentation. Later on, it will be obvious that projects that use deno do not require much CI/CD.

The source code will be hosted on gitlab. There’s no special reason why gitlab and not github.

It is expected that both developpers and users have deno installed on their machine.

Most trivial CLI

The most trivial CLI can be written in a single file. No need for a package.json or build script. See gitlab for the repository.

// file: mod.ts
import { Command } from "https://deno.land/x/cliffy@v0.25.7/command/mod.ts";


function helloWorld(options, ...args) {
    console.log("hello world");
}

await new Command()
  .name("deno-cli-example")
  .version("0.1.0")
  .description("Example of a deno CLI.")
  .action(helloWorld)
  .parse(Deno.args);

Running locally is very easy: deno run mod.ts.

User interaction

Already, users can run and install the application. The only requirement is to point to the raw file. On gitlab raw files have a url defined by <project_url>/-/raw/<branch-name>/<path-to-file>. Unfortunately, these urls are a bit long.

To run the program without installing it, anyone can use the below command.

▶ export BASE_URL=https://gitlab.com/all-dressed-programming/deno-cli-example/
▶ deno run ${BASE_URL}-/raw/most-basic-cli/mod.ts
hello world

Installing the program is very similar. When running the below command, deno will create an executable in the $HOME/.deno/bin/ directory and make it executable. It is the user’s responsibility to add $HOME/.deno/bin to his/her PATH.

▶ export BASE_URL=https://gitlab.com/all-dressed-programming/deno-cli-example/
▶ deno install ${BASE_URL}-/raw/most-basic-cli/mod.ts
▶ $HOME/.deno/bin/most-basic-cli
hello world

Note About Security

Deno is by default secure. Users need to grant permissions for a deno program to use a particular resource. On this, deno is very weby; a deno program runs in the same sandbox as a website visited on google chrome.

Consequently, out of the box, deno would not allow a program to read/write any files. For instance, the beneath command will ask the user for explicit permission to write to /dev/null.

▶ export BASE_URL=https://gitlab.com/all-dressed-programming/deno-cli-example/
▶ deno run ${BASE_URL}-/raw/most-basic-insecured-cli/mod.ts
⚠️  ┌ Deno requests write access to "/dev/null".
    ├ Requested by `Deno.writeFile()` API
    ├ Run again with --allow-write to bypass this prompt.
    └ Allow? [y/n] (y = yes, allow; n = no, deny) > 

Upgrading the software

Looking at what deno does, it is easy to understand the upgrade process. When running deno run https://gitlab.com/my-branch/my-file.ts deno runs the file at the precise URL. When the file changes, the program will run the new code. Using a url that points to file on a branch, and not at a precise git-hash, makes upgrade automatic; each user will use the most up-to-date version of the CLI.

When installing a program, deno uses the same command as deno run. An installed deno program is nothing else than a deno run command wrapped in a bash file.

▶ cat $HOME/.deno/bin/most-basic-cli 
#!/bin/sh
# generated by deno install
exec deno run 'https://gitlab.com/all-dressed-programming/deno-cli-example\
/-/raw/most-basic-cli/mod.ts' "$@"

The entire CI/CD required to maintain a deno CLI can then be summarize as keeping good branching habbits. Visibly, a protected main/master branch where tested pull requests are merged into should be good enough for the majority of the use cases.

Adding State To The CLI

SQLite

In 2023, a very good solution to keep the CLI state is to use an SQLite database. The stability and reliability of SQLite are not to be discussed anymore. Keeping the state in an SQLite has multiple advantages

1. the state is structured into typed SQL tables.
2. very hard to get corrupted state.
3. it is easy to migrate state when software changes.
4. file is easy to work with for developers but nearly impossible to view or alter by none technical users.

The argument here is not that SQLite is the only good solution to persist a CLI state. The argument is that SQLite is a good solution where many other good alternatives exists. Nonetheless, the author of this blog firmly believe in the superiority of SQLite in the vast majority of the situations.

A CLI with SQLite for state

Adding state to a CLI adds some complexity to the code. There is a very cool deno package to work with SQLite. It uses a SQLite binary compiled to webassembly.

The below code is a modified version of the “Most trivial CLI” above. It implements an entire CLI with state stored in SQLite and calls https://httpbin.org to memic calling an external service. The entire CLI is still contained in a single file. The CLI provides two commands: set-properties --username <username> is used to set a username to the state. call-service --username [username] will call https://httpbin.org/get with username as parameter, if no username is passed in the command line, the latest value stored in the state will be used.

import { Command } from "https://deno.land/x/cliffy@v0.25.7/command/mod.ts";
import { DB } from "https://deno.land/x/sqlite/mod.ts";
import {join} from "https://deno.land/std@0.178.0/path/mod.ts";

// for the demo, we will place the state in the /tmp instead of $HOME
const HOME_DIRECTORY = '/tmp';
const CLI_STATE_DIRECTORY = join(HOME_DIRECTORY, 'deno-cli-example');

// the CLI will need read and write access to /tmp/deno-cli-example directory
await Deno.mkdir(CLI_STATE_DIRECTORY, {recursive: true});
const MAIN_DB_FILE = join(CLI_STATE_DIRECTORY, 'state.sqlite');


/**
 * Create tables in the database.  This function is a
 * trivial data migration algorithm.
 */
function initDB(){
  const db = new DB(MAIN_DB_FILE);
  const init_db = `
  CREATE TABLE IF NOT EXISTS usernames (
      id INTEGER PRIMARY KEY AUTOINCREMENT,
      username TEXT,
      creation_ts DATETIME DEFAULT CURRENT_TIMESTAMP
  )
  `;
  try {
    db.execute(init_db);
  } finally {
    db.close();
  }
}

/**
 * Returns the latest username saved in the state.
 */
function getUsernameFromDB(){
  const db = new DB(MAIN_DB_FILE);
  try {
    return db.query(`SELECT username 
                     FROM usernames 
                     ORDER BY creation_ts 
                     DESC LIMIT 1`)[0]
  } finally {
    db.close();
  }
}

/**
 * Set the passed username in the database.
 */
function setUsernameInDB(username: string){
  const db = new DB(MAIN_DB_FILE);
  try {
    return db.query("INSERT INTO usernames (username) VALUES (?)", [username])
  } finally {
    db.close();
  }
}

function getUsernameOrDefault(name?: string): string{
  return name || getUsernameFromDB() || 'no username set yet.';
}

/**
 * Calls httpbin.org/get with the username as argument.
 */
async function getFromService(options){
  const username = getUsernameOrDefault(options.usermame);
  const encodedUsername = encodeURIComponent(username);
  const url = `https://httpbin.org/get?username=${encodedUsername}`;
  const response = await fetch();
  console.log(await response.json());
}

async function setProperties(options){
  if(options.username){
    setUsernameInDB(options.username);
  }
}

initDB();
await new Command()
  .name("deno-cli-example")
  .version("0.1.0")
  .description("Example of a deno CLI.")
  .command('call-service')
  .description('Example of a real use case. call-service will called '
                + 'https://httpbin.org/get with username passed as parameter.')
  .option("--username <username>", 
          'username to use in the service call. If no username specified, the '
           + ' persisted username  will be used.')
  .action(getFromService)
  .command('set-properties')
  .description('Persist properties to state. Persisted properties will then '
                + 'be used as default in other commands')
  .option("--username <username>", "Set the default username property.", 
          {required: true})
  .action(setProperties)
  .parse(Deno.args);

Installing And Running A Full-Fledged CLI

The above CLI imitates what CLI looks like in production. It uses the file system to persist state and calls a remote service.

Using this CLI without installing it is a bit more complicated as we need to specify each permission independently.

▶ export BASE_URL=https://gitlab.com/all-dressed-programming/deno-cli-example/
▶ deno run \
       --allow-read=/tmp/deno-cli-example \
       --allow-write=/tmp/deno-cli-example \
       --allow-net=httpbin.org  \
       ${BASE_URL}-/raw/cli-with-state/mod.ts \
       --help

  Usage:   deno-cli-example
  Version: 0.1.0                                                 

When installing the CLI, the user can allow permissions once at install time and never bother about it afterward. Internally, deno will use the deno run command with the permissions specified during installation.

▶ export BASE_URL=https://gitlab.com/all-dressed-programming/deno-cli-example/
▶ deno install \
       --allow-read=/tmp/deno-cli-example \
       --allow-write=/tmp/deno-cli-example \
       --allow-net=httpbin.org  \
       ${BASE_URL}-/raw/cli-with-state/mod.ts
▶ $HOME/.deno/bin/cli-with-state --help 

  Usage:   deno-cli-example
  Version: 0.1.0

Conclusion

If we accept the requirements developped in this blog, we can accept that deno is an acceptable solution to write CLIs in 2023. Perhaps it is not the only solution, but it looks much better then all the alternatives I have used in the pass.

Personal Thoughts

When starting to write this blog, I did not expect deno to be such a good fit for CLIs. I was particularly impressed by the ease of upgrading deno applications. I was expecting the need to use some hacks there but, as we have all seen, none were required.

In practice, the only rough edge I see is the necessity to install and keep up-to-date deno on other’s people computers. The rest should be a walk in a park.

Point By Point Retrospective