All posts by TIM

TIM’s Tip: Estimating a Data Migration exercise

Maybe you are thinking, what’s so hard, I just dump the data out of system 1 and insert it into system 2, maybe, maybe not.

How long will it take to migrate your data from one or more systems to another system?

It really depends on a few factors:

  1. How easy it is to export data from the source system(s)?
  2. How easily data can be imported into the destination system(s)?
  3. The volume of  data.
  4. The number of objects and fields involved.
  5. Do you need to transform the data before importing into the new system?  and if so, how complex is the transformation?
  6. Are their data dependencies, ie object 1 must be created before uploading object 2?
  7. Do you need a data cleansing exercise to remove/fix bad data and merge duplicates?
  8. If you need to clean the data, should it be loaded into an intermediary data store to make that process easier.

The quality of your data is a really important factor in how long the whole process will take.

If you have such a requirement, check out our migration planning guide or feel free to contact us.

How long does it take to integrate Salesforce and Netsuite?

Well, it depends…

Ok, I realise that isn’t helpful, so here’s an example.

First some assumptions/caveats on Salesforce and Netsuite integration:

  1. Your implementations of Salesforce and Netsuite are in reasonable order, customization is ok, we’re just assuming that you haven’t used duplicate field names in Netsuite and if so, that you understand your data and can identify these.
  2. We also assume that you will play a large part in the identifying the fields to be mapped in each system.
  3. Systems will be locked down during development.
  4. Data migration is not part of the exercise.
  5. We’re using an integration tool that has connectors/adapters for each system.

So, assuming the above, here’s an idea of how long we would take to integrate 3 common objects between Netsuite & Salesforce, ie Account-Customer, Contact-Contact and Opportunity-Opportunity or Opportunity-Invoice.  Our approach is rapid not reckless:

Can we do it quicker?

  • It depends on the assumptions above and how quickly you can work with us to complete the job.

What typically impacts the time-line?

  • Field mapping detail and complexity
  • A clearly defined and well understood business process
  • System availability, People availability (sign-off, testing)
  • Data cleanliness (see this blog on the impact of data quality)

Want to know more, just ping us and we can quickly give you a scope of how long it would take.  You can also check out our We Did section on our website for some examples of recent integrations we have built.

Parature Integration Tip: Field Dependency

Let me share with you about what I have learned in Parature Integration.

Parature offers a great feature that allows you to define field dependency. You can control the visibility of the options based on the controlling custom field.

For example, Operating System is the controlling custom field and Operating System Type is the depending custom field:

  • If you choose Windows, you can only see XP and Vista
  • If you choose Linux, you can only see Redhat and Ubuntu

This is the document returned from Parature API.

Interpretation:

  • Custom field 21 is depending on custom field 20
  • If option 1 (Windows) is selected, enable custom field 21 and its options (10 & 11)
  • If option 2 (Linux) is selected, enable custom field 21 and its options (12 & 13)

If you want to integrate Parature with other systems, all this dependencies may give you headache. You may try this parameter enforceRequiredFields=false to bypass this constraint.

About to integrate existing systems, how’s the quality of your data?

When we kickoff our integration projects we run through a checklist with the client.

If they have been running the systems for a period of time, we always review the quality of the data, we call it a Data Quality Assessment (someone in marketing came up with that, not me, ok).

Usually the client asks:

Why do I need a Data Quality Assessment?

Consider this:

  1. Have you been using the systems involved for some time?
  2. Have you tried to keep them in sync manually? For example: Create Company ABC in both Salesforce and RightNow manually.

If you have, then our advice is to assess the quality of the data before investing in integrating the systems. If the data isn’t cleaned now, we’ll increase the problem once the systems are integrated and bad data is being synchronized.

What’s involved in a Data Quality Assessment?

This involves profiling your data to review if the data is ‘clean‘ , ie do you have excessive duplication and can we merge records to clean this quickly.

What if I don’t clean the data?

Consider the following example:

1) The process for Account/Contact creation in Salesforce is a manual process done using the information in Right Now:

Click to view full picture

2) Given the manual data entry it’s possible for the following to exist:

Click to view full picture

3) When we integrate we will end up with RightNow Case and Opportunity data linked to the wrong Salesforce Accounts.

This is what would occur after running the integration process on the above data:



Click to view full picture

Looks messy right? Definitely something for you to think about before connecting the systems. IMO some type spent up-front will save a lot of pain and $ in the long-term.

It would be like eating health food for a week.

TIM’s Tip: Integrate your Support System with your Issue Tracking System

Integration between systems is not only bringing data over from one system to another. It is an art to get two very different systems working seamless as one. In addition to having your data synchronized in both systems, you can also get a lot of business value from it.

Let’s take a Support System and an Issue Tracking System as an example. You may have a different combination of the following:

Whatever support system and issue tracking system you are using, you will get similar advantages if you integrate both systems together seamlessly:

#1 Use support volume to prioritize bug fix

Development teams can see the impact a bug has caused and prioritise accordingly. You can link one bug with many support cases together.

#2 Enhanced Communication between Support Engineer and Developer

click to view full picture

Support engineer and developer have a dedicated channel to chat. The content is hidden from the support system and the best thing is you don’t have to create 2 logins for everyone!

#3 Support Engineer to provide more details of the bug

When support engineer goes through the bug verification process, they can fill in useful information and that data will be synchronised over to issue tracking system. For example:

  • Environment: Operating system, browser, JVM version
  • Steps to reproduce
  • etc

All these can be automated by the integration process.

There are more benefits, but I can’t mention them all. If you are interested to find out how to do that, feel free to contact us.

Note: Special thanks to Daggy to share these wonderful ideas!

TIM’s Tip: CSV integration

TIM here again – with another Tip, hope it’s helpful.

Boss: TIM, I need you to integrate CSV to Salesforce.
TIM: CSV integration? How hard can that be? I can do it in one day.

A day after…

Boss: TIM, have you completed that CSV integration?
TIM: Hmmm, Boss, I need more time to do this…  (credibility completely shot at this point)

You may be thinking like me. CSV import? How hard can it be. Well it depends, not all CSV imports can be uploaded easily.  Let me give you an example:

1) David owns a company that sells power tools
2) He would like to export sales from his home grown accounting system and import them into Salesforce

This is what the CSV export looks like:

In this scenario, you may need an advanced tool such as Boomi to work out:

  • Line 1 to 4 is Order #1 and Line 5 to 9 is Order #2
  • Order #1 has only 1 item. Order #2 has 3 items

A lot of people think that CSV integration is simple and it’s just data/field mapping – I wish.

Apart from using the right tool, you also need to have the right method of doing it. Here’s a few things I’ve worked out that you need to know:

  1. How the CSV file will be provided? Email? FTP? Read from a local folder?
  2. How to handle rejected data? Email or write into a file?
  3. Understand what objects are involved during the CSV integration.
  4. Understand all mandatory fields in all related objects.
    • If the column is empty in the CSV, do we use a default value or write those records into a file?
  5. Understand the data format for all fields. For example, the date format for the target system may be yyyy-MM-dd’T’HH:mm but your CSV date format is yyyyMMdd HHmm.
  6. Understand the data constraints of each field. For example, First Name field can only accept 100 characters.
  7. Understand the data type of the field. For example, you may need to use a cross reference table for a select list.

Quite a bit to think about, I definitely need a Pina Colada now.

TIMs Tip: Need to get data in & out of Salesforce?

TIM here again with another tip – don’t tell the boss, he thinks I’m working.

Need to sync or integrate Salesforce data with your other systems?

Maybe you want to sync:

  • Accounts/Contacts, Invoice/Sales Order data, payment information, Products and inventory.

So, what are your options?

It depends on what type of  data sync or integration you need and the applications involved, ie:

  1. Do you need real-time or near real-time sync of your data?
  2. Does the data go one way in or out of Salesforce or are you wanting bi-directional updates?
  3. Do you want the whole process automated or are you ok with some manual data loading?
  4. Do you have the budget to integrate?
  5. Does your Salesforce license allow access to the force.com Web Services API?
  6. Is there an existing integration tool that has Connectors/Adapters to your other applications?
  7. How customised are your applications?
  8. Are there any pre-built integration solutions that solve your problems?

Whew, I’m tired just thinking about it.

Let’s take a look at a couple of scenarios:

Some manual work is ok

A good option here is to use one of the Data Loader tools:

  1. http://wiki.developerforce.com/index.php/Apex_Data_Loader
  2. http://sites.force.com/appexchange/listingDetail?listingId=a0330000005kRLFAA2
  • Export your data from your application and use the Data Loader to map this data and upload it into Salesforce.
  • You can also use the tool to export from Salesforce objects into a database or CSV files, you can then manipulate the data and upload into your target applications.

You need the whole process automated

This is where you will need to plan a bit further before you invest in automating the solution.

Do you know your use-cases?

  • You will need to be clear on the process or processes involved and what should be done in the event of failure.  An example of this would be:
  • You want Closed/Won Salesforce Opportunities to generate Invoices in your Accounting system automatically.

You also need to consider any limitations of your Accounting system and if your Products are inventoried

  • ie what will happen if there is no inventory
  • how should the integration process handle such a failure

A good first step here is to take a look at our integration planning guide.

Ok, so you’ve gone through the planning stage and now you want to automate your solution, what tool to choose?

You have a lot of options and the Appexchange is a good place to start.

Some of the key vendors represented include Boomi, Informatica, Pervasive, Cast Iron, Jitterbit and the list goes on.  If you’ve read any of our We Dids you’ll know we are big fans of Boomi and Talend to migrate, manipulate and integrate data.

I’ll cover some of these tools in another blog, but for now, I need a Pina Colada.

TIM’s Tip: bi-directional data integration

TIM here again – with another Tip, hope it’s helpful.

In my experience integrating various SaaS applications such as Salesforce, Netsuite, Parature, RightNow, JIRA, etc,  bi-directional integration is a popular requirement.

So, why use bi-directional integration? Consider a bi-directional update between the Salesforce and RightNow Account Objects as an example:

  • When the record in Salesforce is updated, the linked record in RightNow will also be updated  ie it works both ways.

Now, this isn’t ideal for all systems and mostly suitable for real-time integration. There are a few things to be aware of:

  • You need a mechanism to stop an endless update loop. ie:
  1. A Record updated in Salesforce triggers the integration to update the record in RightNow.
  2. RightNow’s updated record will trigger the integration to update Salesforce’s Record and on and on – you get the idea.
  • In this scenario you should define which fields/data is the Master.
  • Make sure you are checking for timestamps to avoid the scenario where out of date data will update what was the correct data.

Time for a Pina Colada now I think.