EdgeRouter - How can I add new packages and features to EdgeOS?

Overview


Readers will learn how to add new features to the EdgeOS core routing platform. We will look at the process by example of community contributes vyatta-cron package (you can obtain a copy at https://github.com/SO3Group/vyatta-cron).

Vyatta/EdgeOS is based on Debian. Therefore it inherits its packaging system. To simplify the process I created a dummy package:https://github.com/SO3Group/vyatta-dummy

Choose a name for your package. Traditionally the names start with "vyatta". If you want to create separate packages for operational and configuration commands, call them "vyatta-op-*" and "vyatta-cfg-*". Our package is small, so we won't split it. As it's a front-end for cron, would be reasonable to call it "vyatta-cron".

Steps


Install Debian Squeeze, e.g. on a virtual machine.

Install the following packages:

sudo aptitude install build-essential devscripts debhelper autotools-dev autoconf fakeroot automake

 
Create a directory named "vyatta-cron" and copy the skeleton from vyatta-dummy there (everything apart from .git, we will init git from scratch). Init git repository with

git init

 Add proper user and email to ".git/config".

[user]
        name = J. Random hacker
        email = jrhacker@example.net

Config for remotes depends on your setup. For github it's like:

[remote "origin"]
        fetch = +refs/heads/*:refs/remotes/origin/*
        url = git@github.com:YOURUSER/repo-name.git

Package structure


There are several subdirectories and many files.

/debian/            # Debian package build specs
    /control        # Package metadata
    /rules          # Package makefile
    /copyright
    /README
/changelog /scripts/ /templates-cfg/ /templates-op/
AUTHORS
Makefile.am
configure.ac

configure.ac

This is what produces the configure script. The only thing you may want to change is the AC_INIT part.

AC_INIT([vyatta-cron], VERSION_ID, [maintainers@so3group.net])

 Insert your package name in place of "vyatta-cron" and your email.

/debian

First, edit "debian/copyright" and "debian/README". They are straightforward, replace the original name and descriptions with yours.

debian/changelog

Replace its contents with your "initial release" entry. Otherwise it will cause name conflict at build time if you keep an entry with old package name there.

debian/control

Then, the "debian/control". This file defines your package name, descriptions, dependencies, and so on. This is what it looks like in my package:

Source: vyatta-cron
Section: contrib/net
Priority: extra
Maintainer: SO3 Group <maintainers@so3group.net>
Standards-Version: 3.9.1
Build-Depends: debhelper (>= 5), autotools-dev, autoconf

Package: vyatta-cron
Architecture: all
Depends: vyatta-cfg-system,
 vyatta-cfg,
 vyatta-op,
 cron,
 ${misc:Depends}
Description: Vyatta task scheduler configuration
 Vyatta task scheduler configuration utiliites, templates and scripts.

What you need to change:

  1. "Source:". It's source package name, which mostly affects the name of source tarball generated along with .deb at package build. Usually it's the upstream package name, but we are our own upstream, so the same to your package name.
  2. "Maintainer:". You and your contact email.
  3. "Package:". It's the name binary package will have.

"Architecture: all" means the package contains only platform-independent files. If your package only has shell and Perl scripts, it's what you need.

debian/rules

For simple cases there is no much to change there. Find the line that says "PACKAGE=" and replace the name with your package name. In our case:

PACKAGE=vyatta-cron

 Now you should verify your package builds do the following from your package directory:

debuild -us -uc

 "-us -uc" means "do not sign".

If it builds and produces a .deb in the upper level directory, you did everything right. Now to the actual code.

Define the commands

First, think what your commands will be. In most cases it's a good idea not to bind to the underlying implementation names and structure too close, think of entities you are adding, not specific backend config statements.

We wil start with configuration commands, those that are executed after "configure" and modify the config.

For cron we need the only entity: task. We need to be able to set the following properties for it: execution interval, executable path, and arguments passed to the executable. Why to split executable and its arguments? This way we'll be able to find out if the executable exists and save the user some time on troubleshooting. Also, arguments are likely to change more often than the executable path, so the user will not need to enter executable name over and over again.

For users not familiar with UNIX we may provide easy syntax for time intervals (every 10 minutes, every hour etc.). However, it's a good idea to provide seasoned UNIX expers with opportunity to enter the full cron time spec.

So, let's convert these considerations into command tree:

system
    task-scheduler
        task <name>                          # Task name
            cron-spec <UNIX cron time spec>
            executable
                arguments <arguments string>
                path <path to executable>
            interval
                <int32>[mhd]                 # m for minutes, h for hours, d for days

Now we need to tell Vyatta what commands we want. Vyatta uses "templates" for it. Templates consist of directory tree where directory named match configuration path names, and special files named "node.def" that define comand behaviour.

Go to our package "templates-cfg". Remove the sample directory and create the following directory tree:

templates-cfg/
    system/
        task-scheduler/
            node.def
            task/

 Every directory tree level needs a "node.def", even if it doesn't define any behaviour (otherwise the system ignores that directory). For "system/" we inherit that file from the system (as it's already defined by vyatta-cfg-system package), for "system/task-scheduler" we need our own.

Edit "system/task-scheduler/node.def" and write there:

help:  Task scheduler settings

 That help text will be displayed in completion (and tell the system it's a template directory).

Now to the tricky part. We obviously need to be able to create multiple tasks. How? EdgeOS/Vyatta calls nodes that may have multiple instances and serve as containers for other nodes "tag nodes". There is special template syntax for it.

task/
    node.def
    node.tag/
        crontab-spec/
            node.def
        executable/
            arguments/
                node.def
            path/
                node.def
        interval/
            node.def

Everything we put under that "node.tag" dir will appear in every task instance. Now we need to create a node.def to notify Vyatta "task" is a tag node and how to handle it.

Edit "system/task-scheduler/task/node.def":

tag:
type: txt
help: Scheduled task
val_help: Task name
priority: 999
end:
    if [ ${COMMIT_ACTION} == DELETE ]; then
        sudo rm -f /etc/cron.d/vyatta-crontab
    else
        sudo /opt/vyatta/sbin/vyatta-update-crontab.pl --update || exit 1
    fi
  1.  "type:" defined allowed value type, the system will refuse to set values of other types. The common types are: "txt" — string with no format checking, "int32" — unsigned integer, "ipv4" — single IPv4 address, "ipv4net" — IPv4 CIDR, "ipv6" — single IPv6 address, "ipv6net" — IPv6 CIDR.
  2. "tag:" is what tell Vyatta it's a tag node.
  3. "help:" is the help string displayed in completion.
  4. "val_help:" is the help string displayed in second level of completion (double Tab).
  5. "priorty:" defines where in commit process out subtree is processed. "999" basically means "after everything". We do this because user scripts are likely to rely on consistent system state.
  6. "end:" is where we define the behaviour. It's a Bourne shell snippet that calls external scripts or perform actions to configure the system according to set values directly (e.g. if you want just to set sysctl value, you don't need external script likely).

"end:" is handy for the cases all configuration is handled by external script, like in our case. For the cases we need different actions on creation, deletion, or value change, there are other statements:

  • "create:" — executed when you create node that didn't exist.
  • "update:" — executed when node value is changed.
  • "delete:" — executed when node is deleted.
  • "begin:" ­— executed before any other statements.

Now we also need to write other node.def's. Most of them are not interesting, I'll describe only "executable/path" node here. It's interesting because its value can be checked easily at set time, and we'll do it:

ype: txt
syntax:expression: exec
    "if [ ! -x $VAR(@) ]; then \
        echo \"File $VAR(@) does not exist or is not executable\"; \
        exit 1; \
    fi; "
help: Path to executable

 Note the "syntax:expression:". It's a shell statement (yes, statement, so \'s are necessary if you break lines). If it returns non-zero value, set command fails.

Operational mode commands

There are way simpler than configuration mod. Just create directories and put node.def's with contents like:

help: Help string
run: Command to execute

 In cron we have a script that calls "/etc/init.d/cron restart" if "system task-scheduler" configuration exists.

scripts/

Put your scripts there.

The cron script is pretty straightforward, you can find the config API for shell here: http://vyattawiki.net/wiki/Cli-shell-api , and here for Perl:http://vyattawiki.net/wiki/Vyatta::Config

Makefile.am

Now we need to tell the build process what files to take and where to place them. If you are not familiar with autotools, think of it's as magic (and learn about autotools).

Edit "Makefile.am". First, we need to place our configuration scripts to "/opt/vyatta/sbin" dir, and we write:

sbin_SCRIPTS = scripts/vyatta-update-crontab.pl

 Service restart script goes to "/opt/vyatta/bin/sudo-users" and we write:

bin_sudo_users_SCRIPTS = scripts/vyatta-restart-crond.sh

Revision control

Don't forget to add your successfull changes to git.

Initially, do:

git add *
git commit

Then you can add just files you changed to let git know you did:

git add scripts/*
git add Makefile.am
git commit

Avoid going "git add *" after you built a package as you'll get a lot of autogenerated garbage in the repository! Always add just files you changed, or do "git clean" first.

Conclusion

This is the bare minimum you need to create your own package. Look at the wiki pages for API, look at other packages from the GPL archive and experiment, and very soon you will be able to implement nearly whatever you want.

Notes


You can you revision control system of your choice (Mercurial, Bazaar, Subversion etc.), but EdgeOS and Vyatta developers are using git, so if you are going to get your package integrated into the upstream, better use git.