Build distribution tool for SenchaCMD

2016-03-25 tags: sencha cmd extjs bash

Building ExtJS application with SenchaCMD takes about one minute on my 4-cores laptop. In current project we have about 20 ExtJS applications that are included into one main application (an architectural requirement). So, when I am building whole app, I should take 20-minutes coffee break. Sounds not so bad? Not after third cup :)

It happens when your ExtJS application consist of several parts and each of them needs to be builded separatly. Or you have to build several copies of one application for different environments. In my case we have several applications in one SenchaCMD workspace. First of all I disabled unimportant build features, such like theme’s sampling. I wrote small bash script that helps distribute builds to different VMs:

This is Jenkins result for 3 distributed VMs used instead one:

Looks cool?

Repository on GitHub.

Features

Distributing builds ExtJS applications
Manage build-agents from one master
Add/remove build-agent, install requirements (Java, SenchaCMD, …)
Getting list of all applications in folder automatically
Possible to run by Jenkins
Shows build time and build progress.

System requrements

Debian-based system (Ubuntu).

Installation

For start working with this script we do not have to clone whole repository, just copy the main bash script.
Copy dscmd.sh script to your Sencha applications workspace folder with the following command:

$ wget https://raw.githubusercontent.com/antonfisher/dscmd/master/dscmd.sh -O dscmd.sh

Next, apply execution privileges to the file:

$ chmod +x dscmd.sh

And run configuration process:

$ ./dscmd.sh config

During configuration we define an application folder, and a path to SenchaCMD on agents (can use default):

$ ./dscmd.sh config
Build distribution tool for SenchaCMD v0.1.3 [beta]
Master initialization.

Enter path to applications folder (default: 'pages' or previous uses '') [ENTER]: apps/
Found applications in 'apps': app-about,app-dashboard,app-setting

Enter path to SenchaCMD on agents (default: ~/bin/Sencha/Cmd/sencha or previous uses) [ENTER]: 

Saved to .dscmd-config:
APPS_PATH=apps
CMD_PATH=~/bin/Sencha/Cmd/sencha

Usage

To get usage help just run ./dscmd.sh without parameters:

$ ./dscmd.sh
Build distribution tool for SenchaCMD v0.1.3 [beta]
Usage:
  ./dscmd.sh config
  ./dscmd.sh applications-list
  ./dscmd.sh add-agent
  ./dscmd.sh remove-agent [--all]
  ./dscmd.sh agents-list
  ./dscmd.sh agents-test
  ./dscmd.sh build [--all] <application1,application2,...>

Add agents

A build-agent needs to be a Debian-based host (Ubuntu). First of all we need to copy ssh key to each agent. And then run add-agent command and follow instructions:

$ ./dscmd.sh add-agent
Build distribution tool for SenchaCMD v0.1.3 [beta]
Add agent wizard.

Enter agent ip or host (use ',' to add few agents with same username) [ENTER]: 10.3.3.101,10.3.3.102
Enter agent username (default: root) [ENTER]: root
Enter path to SenchaCMD installation script [ENTER]: ~/Downloads/SenchaCmd-6.0.2.14-linux-amd64.sh 
Copy ssh key to agent using ssh-copy-id (Y/n) [ENTER]: 
Apt-get update and upgrade agent (Y/n) [ENTER]: n
Install Java and Ruby (Y/n) [ENTER]: n
Start...

### PART OF INSTALATION PROCESS BELOW ###

Create 'dscmd' folder on [email protected]...
Connection to 10.3.3.101 closed.
Copy SenchaCMD installation script (/home/af/Downloads/SenchaCmd-6.0.2.14-linux-amd64.sh) to [email protected]:~/dscmd ...
SenchaCmd-6.0.2.14-linux-amd64.sh                                                          100%   53MB   7.6MB/s   00:07    
Run SenchaCMD installation script on [email protected]...
Starting Installer ...
This will install Sencha Cmd on your computer.

...

Setup has finished installing Sencha Cmd on your computer.
Finishing installation ...
Connection to 10.3.3.101 closed.
Syncronize directory with [email protected]:/dscmd...

...

Done.

Run distributed build

Our agents are ready, let’s try to build:

Run $ ./dscmd.sh build --all to build all application in applications folder;
Or run $ ./dscmd.sh build applicationName1,applicationName2.

To show the applications list which will be used in case of --all option, use applications-list commad:

$ ./dscmd.sh applications-list
Build distribution tool for SenchaCMD v0.1.3 [beta]
Applicaitons list will be used for build:
about,gettingStarted,actionEdit,accessControl,actionDelete,actionDeleteDS,settings,summary

Build output:

$ ./dscmd.sh build --all
Build distribution tool for SenchaCMD v0.1.3 [beta]
Build applications:

[build   1/19: about         ] run build 'about' on [email protected]
[build   3/19: gettingStarted] run build 'gettingStarted' on [email protected]
[build   6/19: actionEdit    ] run build 'actionEdit' on [email protected]
[build   1/19: about         ] Syncronize local directory with agent...
[build   6/19: actionEdit    ] Syncronize local directory with agent...
[build   2/19: accessControl ] run build 'accessControl' on [email protected]
[build   2/19: accessControl ] Syncronize local directory with agent...
[build   5/19: actionDeleteDS] run build 'actionDeleteDS' on [email protected]
[build   3/19: gettingStarted] Syncronize local directory with agent...
[build   4/19: actionDelete  ] run build 'actionDelete' on [email protected]
[build   4/19: actionDelete  ] Syncronize local directory with agent...
[build   5/19: actionDeleteDS] Syncronize local directory with agent...
[build   6/19: actionEdit    ] Sencha Cmd v6.0.2.14
[build   5/19: actionDeleteDS] Sencha Cmd v6.0.2.14

Agents

In my case I depoyed this list of VMs:

Each of them has the same configuration:

Under hood

Simple diagrams what is happening under hood:

Install agent:
    copy ssh key --> ssh --> apt-get install --> install sencha cmd --> initial rsync

Build application:
    rsync --> sencha app build --> rsync

Conclusion

We have 6 build-agents and now build takes about 6 minutes instead of 20 minutes for 18 applications. It is not bad.

Feel free to ask me any questions.