Author: Andrey Dubinin, Software Engineer
In the process of software development, it’s not only hard to do without bug tracking systems, but it’s virtually impossible. Redmine system is especially convenient for such purposes.
It has everything we need:
- Keeping track of tasks’ status
- Tasks grouping in tracker
- Inter-project discussion if necessary
- Account of time spent by the employees and types of their activity
However, a bug tracking system can be more effective when incorporated with CRM software. In this guide, we will tell how to integrate SugarCRM with Redmine for building reports on the time spent on the tasks assigned by the customer at SugarCRM portal.
By the way, at our website, you can find a vast library of SugarCRM video tutorials and webinars. Various useful enhancements, solutions to the most important tasks are available here: https://integroscrm.com/sugarcrm-video-tutorials-and-webinars/.
Task
Obviously, data collection should be an ongoing process in order to respond promptly to negative changes. However, big data hinder manual carrying out of the task.
So, our task on SugarCRM integration with Redmine is the following: we need to transmit customer’s tasks and related comments from SugarCRM portal to Redmine. Also it is needed to automate the process of transmitting time spent to SugarCRM from Redmine in order to build reports within SugarCRM and submit them to the customer.
SugarCRM is famous for its flexible Rest API, using it we are going to get data. The things are different about Redmine – Redmine also has its own API, but it’s not that flexible and quick-operating.
To meet all the integration requirements we need a utility which will be able to “communicate” with Rest services and pass data between two systems. We will apply NodeJS, which uses JavaScript language for development and can be extended by various plugins.
We will need the following plugins:
- Mssql. The plugin through which we will transmit data to Redmine database.
- Mkdirp. We will create catalogues with utility reports.
- Underscore. A set of utilities to simplify the development.
- Winston. With this plugin we will log the utility working process.
- Node-Rest-Client. REST client for communication between Redmine and SugarCRM.
- Async. With this plugin we will ensure asynchronous actions to speed up the synchronization process.
Let’s get started
First and foremost, let’s install NodeJS. The information on its installation can be found on the Internet or on the official website. Let’s get down to creating our project. Firstly, let’s generate npm package: open the console, go to the catalogue where our project will be stored and run the npm install command. In the process we will be offered to insert name, version, comment, author, etc. to generate projects’ file (package.json).
After completion we open package.json file and add our plugins for utility creation. As a result, we will get the file, as shown below:
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 | { "name": "RedmineSync", "version": "1.5.1", "description": "Redmine SugarCRM Sync", "author": "Andrey Dubinin", "main": "start.js", "license": "ISC", "repository": { "type": "git", "url": "" }, "dependencies": { "mssql":"*", "mkdirp":"*", "underscore": "*", "winston": "*", "node-rest-client": "1.0.0", "async": "*" } } |
Let’s go back to the console. We will install plugins to the project npm install command. And finally, let’s create the utility itself.
In the process we will have to create utilities for integrating SugarCRM and Redmine. I called them sugar-rest.js, redmine-db.js and redmine-rest.js. They store functions ad decorators for receiving and sending data. The key issue here is authorization in SugarCRM, whereby there is only one session per a user, which limits the user in time. Which is why it is necessary to check the session availability before carrying out this or that task in SugarCRM. The example is given below:
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 | /* * Take data from SugarCRM * params * array options: Function inserted params * function callback: Callback * */ getData: function (options, callback) { var self = this; //Call check user session this.refreshToken(options.sugarAuth, function (sugarAuth) { options.sugarAuth = sugarAuth; //Take REST service var client = self.getRestClient(); //Generate header params var args = { headers: { "Content-Type": "application/json", "OAuth-Token": sugarAuth.token } }; //Send request in to SugarCRM and calculate response client.get(restConfig.baseURI + options.uri, args, function (data) { var result = {sugarAuth: sugarAuth}; if (_.has(data, 'error')) { result.error = data.error_message; } else { result.data = data; } callback(result); }).on('error', function (err) { logger.error(err); }); }); }, |
And the function that actually checks the session in SugarCRM, restores it in case of a loss or logs the error if the restoring fails.
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 | /* * Check connection and session data * array sugarAuth: Session params * function callback: Callback * */ refreshToken:function(sugarAuth, callback){ //ping connection this.ping(sugarAuth.token, function(err,data){ if(data !== undefined && data == 'pong'){ callback(sugarAuth); return; } var sugarRest = require('./sugar-rest').PxSugarRest; //If session is end or lost then refresh session sugarRest.refresh(sugarAuth.refreshToken, function (data) { if(!(typeof data === "object" && _.has(data,'access_token'))){ //Auch if session lost sugarRest.authenticate(function (err, authData) { if (err) { logger.error(err.toString()); return; } sugarAuth.token = authData.access_token; sugarAuth.refreshToken = authData.refresh_token; callback(sugarAuth); }); }else{ sugarAuth.token = data.access_token; sugarAuth.refreshToken = data.refresh_token; callback(sugarAuth); } }); }); }, |
The things are similar with a set of utilities for Redmine, the utilities themselves serve to simplify the calls for receiving and sending data to the servers. Further we will call them when creating the scenario of synchronization sequence.
Now we need to implement the algorithm of by-turn synchronization initiation by the utility run. I have 3 synchronization files available: sync-cases.js, sync-notes.js и sync-time-entries.js. The sequence of running the synchronization is presented in the form of sequence diagram, shown below:
Next we have to implement the algorithm of connection to servers and data processing. The algorithm example is provided in the sequence diagram below. Similarly is performed the sync process of the Spent time and Comments.
Key issues in the development process
Big data issue
When choosing a big volume of data the system can “freeze” or crash due to memory lack or “freezing” on the SugarCRM or RedmIne server side.
Solution:
In SugarCRM WEB service there is an opportunity to get these records in the number of 50 or more. It is done via GET /<module> request with parameters stated in the table below.
| Name | Type | Description |
| max_num | Integer | Max number of records to be returned by the request. 20 by default. |
| offset | Integer | Number of records that should be skipped. 0 by default. |
Issue with the user’s session in SugarCRM service
The system can report errors related to Token. Token is a user’s session key to SugarCRM service. The list of reasons the error may occur:
- Somebody logged in SugarCRM under utility user, and the session has changed.
- The second copy of utility is run.
- The session is over (by default SugarCRM provides 1 hour per session).
Solution:
In order to solve the mentioned problem, the Token check has been implemented during each access to SugarCRM via synchronization utility. This enhancement has excluded Token malfunction, but the processes take more time too. See the diagram below.
Advantages
1. 4 logging levels that inform on the process: debug, info, warn and error, as well as logs that alert of utility fatal errors.
2. Possibility of various utility configuration. The list of the key ones is given below:
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 | //Data of sync start in ISOString format dateStart: '2015-09-20T00:00:00.000Z', //Interval between iterations in ms iterationInterval: 1000, //Number of records checked at a time //(-1)Check all records //(0)Not check //(100)Check by 100 e.g., any non-negative number can be stated //Accounts //Global log storing //logDir: 'C:/SyncLogs', //Local log storing logDir: './logs', casesCount: 200, //Notes notesCount: 200, //Time spent timeEntryCount: 200, //Sugar server settings sugar: { //Used details rest: { baseURI: 'http://<Sugar url>/rest/v10/', user: 'login', password: 'pass' } }, //RedMine settings redmine: { //data base db: { user: 'user', password: 'pass', server: '<ip address>', database: 'db name' }, //User details rest: { baseURI: 'http://<url>/', user: 'user', password: 'pass', apiKey: 'api key code' }, //Name of referral code in RedMine customFields: { contractRefCode: 'SugarCRMContractRefCode' } } |
3. The utility is realized on NodeJS, which instead of a traditional parallelism model based on streams, is implemented based on event-focused system. The model has been chosen due to its simplicity, low overheads and high performance.
Disadvantages
1. Relatively long data processing time.
2. Restricted number of updatable fields.
3. Limited functionality: FB «Tasks», «Comments» and «Spent time».
How the SugarCRM integration can be extended
The SugarCRM integration with Redmine is based on the procedure model; it can be changed to object model, which will give a possibility to update more fields, to organize code in a better way, to simplify enhancement and testing.
Add more flexible algorithm which will allow the user add more settings and restrict the functionality less.
That’s it. If you need more advice on SugarCRM integration with Redmine or other software – give us notice.
Other SugarCRM tutorials: