Skip to main content

Importing data from the Composer module

In Varicent Incentives, you can import data into any hierarchy, system, data, custom, or structural table from a file or third party application.

Note

To learn more about importing, exporting, and manipulating data in Composer, take the Importing, Exporting, and Manipulating Data in Incentives Composer course.

Data can be imported from your computer or from Incentives on cloud storage. Files that are imported from your computer with the Save this import as checkbox selected are saved to Incentives on cloud storage.

Incentives uses two types of data connectors to import data into tables:

To learn more about importing data from the Composer module, take the Importing, Exporting, and Manipulating Data in Incentives Composer course.

Importing data from text files

From the Composer module, you can import data into tables from text files.

  1. From the Composer module, find the table that you want to import data into.

  2. Click Import.

  3. Select Text File.

  4. Upload the file by dragging it or browsing for it.

    Note

    Accepted file formats are .txt or .csv.

  5. Click Next.

  6. If the first row of your file contains labels for each column, turn on the First record is a header toggle.

    This indicates that the first row should not be processed as data, as it is a label for each column.

  7. Select the delimiter from the Delimiter drop-down list.

    When importing from a text file, you can choose a special character to import fields that contain a delimiter. You can choose from these delimiters: comma, semicolon, tab, pipe (|), and period. This is useful if you have data that contains a comma but is importing from a comma delimited file. Without some sort of special character, there would be no way to import a field containing a comma from that kind of file.

  8. Select the language of the file you're importing.

  9. Select the date format of the file you're importing.

  10. To see a preview of the file, turn on the Show preview toggle.

  11. Click Next.

  12. Map the columns in the target table, listed under Destination table columns, to the Source file fields in the text file data set.

  13. Select any or all of following options available within the wizard:

    Option

    Description

    Manage fields

    Adds a new field as part of your data import. Click Manage fields, and select either Add new field or Merge fields from the drop-down list. Multiple new fields can be added.Adding new fieldsMerging Fields

    Automatch

    Turns on the Automatch toggle to automatically match columns with identical names.

    Show only unmatched columns 

    Turns on the Show only unmatched columns toggle to list only the unmatched columns.

  14. Click Next.

  15. Select any or all of the following run options:

    Option

    Description

    Update existing rows

    Indicates that existing rows of data should be updated with data contained in the imported rows.

    Run this import now

    Indicates that the import should be run immediately. Clear this checkbox if you want to save the current import and process it at a later date.

    Use incremental import

    Indicates that rows should be imported into the database in smaller batches instead of committing the entire import in one batch. Use this option to process database calls, such as report generation in the Sales Portal, during a large data import.

    Save this import as

    Saves the import so you can run it at a later time using Scheduler. If this option is not selected, the file is not saved to Incentives storage. See the Scheduler Module section for more information on using Scheduler.

  16. Click Finish.

Importing data from Excel files

From the Composer module, you can import data into tables from Excel files.

Tip

Looking for a way to edit your data in Excel and push it directly to Incentives? Try the Varicent add-in for Excel.

  1. From the Composer module, find the table that you want to import data into.

  2. Click Import.

  3. Select Microsoft Excel.

  4. Upload the file by dragging it or browsing for it.

    Note

    The Excel file can be no larger than 2MB. To import Excel files larger than 2MB, use the CLI import tool.

  5. Click Next.

  6. If the first row of your file contains labels for each column, turn on the First record is a header toggle.

    This indicates that the first row should not be processed as data, as it is a label for each column.

  7. Select the worksheet you want to import.

  8. Select the language of the file you're importing.

  9. Select the date format of the file you're importing.

  10. To see a preview of the file, turn on the Show preview toggle.

  11. Click Next.

  12. Map the columns in the target table, listed under Destination table columns, to the Source file fields in the Excel file data set.

  13. Select any or all of following options available within the wizard:

    Option

    Description

    Manage fields

    Adds a new field as part of your data import. Click Manage fields, and select either Add new field or Merge fields from the drop-down list. Multiple new fields can be added.Adding new fieldsMerging Fields

    Automatch

    Turns on the Automatch toggle to automatically match columns with identical names.

     Show only unmatched columns 

    Turns on the Show only unmatched columns toggle to list only the unmatched columns.

  14. Click Next.

  15. Select any or all of the following run options:

    Option

    Description

    Update existing rows

    Indicates that existing rows of data should be updated with data contained in the imported rows.

    Run this import now

    Indicates that the import should be run immediately. Clear this checkbox if you want to save the current import and process it at a later date.

    Use incremental import

    Indicates that rows should be imported into the database in smaller batches instead of committing the entire import in one batch. Use this option to process database calls, such as report generation in the Sales Portal, during a large data import.

    Save this import as

    Saves the import so you can run it at a later time using Scheduler. If this option is not selected, the file is not saved to Incentives storage. See the Scheduler Module section for more information on using Scheduler.

  16. Click Finish.

Importing data from database files

From the Composer module, you can import data into tables from database files.

  1. From the Composer module, find the table that you want to import data into.

  2. Click Import.

  3. Select Database Import.

  4. If you want to use a Text Driver, turn on the Use Microsoft Text Driver toggle and select a file to import.

  5. Click Configure.

  6. Configure your SQL query.

  7. To see a preview of the file, turn on the Show preview toggle.

  8. Click Next.

  9. Map the columns in the target table, listed under Destination table columns, to the Source file fields in the database import file data set.

  10. Select any or all of following options available within the wizard:

    Option

    Description

    Manage fields

    Adds a new field as part of your data import. Click Manage fields, and select either Add new field or Merge fields from the drop-down list. Multiple new fields can be added.Adding new fieldsMerging Fields

    Automatch

    Turns on the Automatch toggle to automatically match columns with identical names.

    Show only unmatched columns 

    Turns on the Show only unmatched columns toggle to list only the unmatched columns.

  11. Click Next.

  12. Select any or all of the following run options:

    Option

    Description

    Update existing rows

    Indicates that existing rows of data should be updated with data contained in the imported rows.

    Run this import now

    Indicates that the import should be run immediately. Clear this checkbox if you want to save the current import and process it at a later date.

    Use incremental import

    Indicates that rows should be imported into the database in smaller batches instead of committing the entire import in one batch. Use this option to process database calls, such as report generation in the Sales Portal, during a large data import.

    Save this import as

    Saves the import so you can run it at a later time using Scheduler. If this option is not selected, the file is not saved to Incentives storage. See the Scheduler Module section for more information on using Scheduler.

  13. Click Finish.

Importing data from Salesforce.com

From the Composer module, you can import data into tables from Salesforce.com.

  1. From the Composer module, find the table that you want to import data into.

  2. Click Import.

  3. Select Salesforce.com.

  4. Type your Salesforce.com username, password, and security token.

  5. Click Connect.

  6. Select the table that you want to import.

    The menu lists all of the Salesforce.com tables that you have access to.

  7. Click Next.

  8. Select the date format of the file you're importing.

  9. To see a preview of the file, turn on the Show preview toggle.

  10. Click Next.

  11. Map the columns in the target table, listed under Destination table columns, to the Source file fields in the the Salesforce.com table.

  12. Select any or all of following options available within the wizard:

    Option

    Description

    Add field

    Adds a new field as part of your data import. Click Manage fields, and select either Add new field or Merge fields from the drop-down list. Multiple new fields can be added.Adding new fieldsMerging Fields

    Automatch

    Turns on the Automatch toggle to automatically match columns with identical names.

    Show only unmatched columns

    Turns on the Show only unmatched columns toggle to list only the unmatched columns.

  13. Click Next.

  14. Select any or all of these run options:

    Option

    Description

    Update existing rows

    Indicates that existing rows of data should be updated with data contained in the imported rows.

    Run this import now

    Indicates that the import should be run immediately. Clear this checkbox if you want to save the current import and process it at a later date.

    Use incremental import

    Indicates that rows should be imported into the database in smaller batches instead of committing the entire import in one batch. Use this option to process database calls, such as report generation in the Sales Portal, during a large data import.

    Save this import as

    Saves the import so you can run it at a later time using Scheduler. If this option is not selected, the file is not saved to Incentives storage. See the Scheduler Module section for more information on using Scheduler.

  15. Click Finish.

Importing data from XML files

From the Composer module, you can import data stored in XML format directly into tables.

  1. From the Composer module, find the table that you want to import data into.

  2. Click Import.

  3. Select XML File.

  4. Upload the file by dragging it or browsing for it.

  5. If the first row of your text file contains labels for each column, turn on the First record is a header toggle.

    This indicates that the first row should not be processed as data, as it is a label for each column.

  6. Select the row node to import data from.

  7. Select the language of the file you're importing.

  8. Select the date format of the file you're importing.

  9. To see a preview of the file, turn on the Show preview toggle.

  10. Click Next.

  11. Map the columns in the target table, listed under Destination table columns, to the Source file fields in the XML file data set.

  12. Select any or all of following options available within the wizard:

    Option

    Description

    Manage fields

    Adds a new field as part of your data import. Click Manage fields, and select either Add new fieldAdding new fields or Merge fieldsMerging Fields from the drop-down list. Multiple new fields can be added.

    Automatch

    Turns on the Automatch toggle to automatically match columns with identical names.

     Show only unmatched columns 

    Turns on the Show only unmatched columns toggle to list only the unmatched columns.

  13. Click Next.

  14. Select any or all of the following run options:

    Run option

    Description

    Update existing rows

    Indicates that existing rows of data should be updated with data contained in the imported rows.

    Run this import now

    Indicates that the import should be run immediately. Clear this checkbox if you want to save the current import and process it at a later date.

    Use incremental import

    Indicates that rows should be imported into the database in smaller batches instead of committing the entire import in one batch. Use this option to process database calls, such as report generation in the Sales Portal, during a large data import.

    Save this import as

    Saves the import so you can run it at a later time using Scheduler. If this option is not selected, the file is not saved to Incentives storage. See the Scheduler Module section for more information on using Scheduler.

  15. Click Finish.

Importing data from Varicent ELT

Importing data into the Composer module directly from Varicent ELT is useful if you want to transform your data in Varicent ELT before adding it to your Varicent Incentives model. You can also save and schedule imports from Varicent ELT.

Important

Before you can import data from Varicent ELT, ensure that you have Varicent ELT enabled on your model.

  1. From the Composer module, find the table that you want to import data into.

  2. Click Import.

  3. Select Varicent ELT.

  4. Select the Varicent ELT pipe output from the drop-down menu and click Next.

  5. If your source data is stored in Incentives:

    1. Under Advanced Options, ensure the Upload from source table toggle is turned on. Click Next.

    2. In the Send data to pipe tab, select the table in Incentives that contains the source data from the Load from source data drop-down list.

      Match the columns in the data source (Data Source Columns) to the columns in the Varicent ELT pipe (Data Tool Columns). Click Next.

    3. In the Map result to table tab, select the Varicent ELT export tool from the Varicent ELT export tool drop-down list.

      Match the columns in the Varicent ELT pipe (Export Tool Columns) to the source you want to import into (Output Columns).

      Tip

      Click Automatch to automatically match columns with identical names.

    4. By default, the Refresh all pipe data sources option is checked. To skip a data refresh and have a shorter data loading time, uncheck the Refresh all pipe data sources checkbox.

  6. If your source data is from a local file:

    1. Under Advanced Options, ensure the Upload from source table toggle is turned on. Click Next.

    2. Click Load from a file.

      Important

      Accepted file formats are .txt, .csv, .xls, and .xlsx.

    3. Choose a previously uploaded file, or upload a new file by dragging it or browsing for it. Click Next.

    4. If the first row of your file contains labels for each column, turn on the First record is a header toggle.

    5. If you're importing a text file, select the delimiter from the Delimiter drop-down list.

    6. If you're importing an Excel file, select the worksheet you want to import.

    7. Select the language of the file you're importing.

    8. Select the date format of the file you're importing.

    9. To see a preview of the file, turn on the Preview toggle.

    10. Click Done.

    11. In the Map result to table tab, select the Varicent ELT export tool from the Varicent ELT export tool drop-down list.

      Match the columns in the Varicent ELT pipe (Export Tool Columns) to the source you want to import into (Output Columns).

      Tip

      Click Automatch to automatically match columns with identical names.

    12. By default, the Refresh all pipe data sources option is checked. To skip a data refresh and have a shorter data loading time, uncheck the Refresh all pipe data sources checkbox.

  7. If your source data is stored in Varicent ELT:

    1. Under Advanced Options, turn off the Upload from source table toggle.

    2. Turn on the Run pipe (transform your data) toggle. Click Next.

    3. In the Map result to table tab, select the Varicent ELT export tool from the Varicent ELT export tool drop-down list.

      Match the columns from the Varicent ELT pipe (Export Tool Columns) to the source you want to import into (Output columns).

      Tip

      Click Automatch to automatically match columns with identical names.

    4. By default, the Refresh all pipe data sources option is checked. To skip a data refresh and have a shorter data loading time, uncheck the Refresh all pipe data sources checkbox.

  8. Click Next.

  9. Select any or all of these run options:

    Option

    Description

    Update existing rows

    Updates any existing rows of data with data that's contained in the imported rows.

    Run this import now

    Runs the import immediately. If you want to save the current import and process it later, clear this checkbox.

    Use Incremental Import

    Imports the table rows into the database in smaller batches instead of committing the entire import in one batch. Use this option to process database calls, such as report generation in the Sales Portal, during a large data import.

    Save this import as

    Saves the import so you can run it at a later time using Scheduler. If this option is not selected, the file is not saved to Incentives storage. See the Scheduler Module section for more information on using Scheduler.

  10. Click Done.

Using the Advanced connection tool to import data from Varicent ELT

In the Composer module, you can access additional data sources from Varicent ELT pipes, or create a new Varicent ELT pipe using the Advanced connection tool.

  1. From the Composer module, find the table that you want to import data into.

  2. Click Import.

  3. Select Advanced connection.

  4. Choose one of the following options:

    1. Use the search bar to search for a pipe in your Varicent ELT model. Move on to the next step to continue with the advanced Varicent ELT connection.

    2. To create a new Varicent ELT pipe, click Create new pipe. Log in to your Varicent ELT account to create a new pipe. To learn more about using Varicent ELT, read this article on building pipes.

  5. Select a pipe from the list.

  6. From the drop-down list, select the Export tool that includes your data.

    Tip

    Click the Show preview toggle to preview the pipe output.

  7. Click Next.

  8. Map the columns in Incentives listed under Data table columns to the Pipe output fields in Varicent ELT.

  9. Select any or all of following options available within the wizard:

    Option

    Description

    Automatch

    Turns on the Automatch toggle to automatically match columns with identical names.

    Show only unmatched columns 

    Turns on the Show only unmatched columns toggle to list only the unmatched columns.

  10. Click Next.

  11. Select any or all of these run options:

    Option

    Description

    Update existing rows

    Updates any existing rows of data with data that's contained in the imported rows.

    Run this import now

    Runs the import immediately. If you want to save the current import and process it later, clear this checkbox.

    Use incremental import

    Imports the table rows into the database in smaller batches instead of committing the entire import in one batch. Use this option to process database calls, such as report generation in the Sales Portal, during a large data import.

    Save this import as

    Saves the import so you can run it at a later time using Scheduler. If this option is not selected, the file is not saved to Incentives storage. See the Scheduler Module section for more information on using Scheduler.

  12. Click Done.

Importing data from Google Sheets

From the Composer module, you can import data into tables from Google Sheets.

Important

Before you can import data from Google Sheets, ensure that you have Varicent ELT enabled on your model.

  1. From the Composer module, find the table that you want to import data into.

  2. Click Import.

  3. Click Google Sheets.

  4. To connect to a new Google account:

    1. Click New connection.

    2. Type a connector name.

    3. Choose a Google account, or click Use another account and enter the account login details.

    4. Click AdvancedGo to varicent.com.

    5. Select all the checkboxes that apply to grant Incentives access.

    6. Click Continue.

    Important

    When you connect to a Google account, all users in the same Varicent ELT organization will be able to view and access that account.

  5. Select a Google account to connect to from the list of connections.

  6. From the Directory drop-down list, select a file to import data from and click Next.

  7. Select the sheets you'd like to include and click Next.

  8. Select the data you'd like to include or obfuscate and click Next.

  9. Click Finish.

  10. Map the columns in the target table, listed under Destination table columns, to the Source file fields in Google Sheets.

    To list only the unmatched columns, turn on the Show only unmatched columns toggle.

    Click Next.

  11. Select any or all of the following run options:

    Option

    Description

    Update existing rows

    Updates any existing rows of data with data that's contained in the imported rows.

    Run this import now

    Runs the import immediately. If you want to save the current import and process it later, clear this checkbox.

    Use incremental import

    Imports the table rows into the database in smaller batches instead of committing the entire import in one batch. Use this option to process database calls, such as report generation in the Sales Portal, during a large data import.

    Save this import as

    Saves the import so you can run it at a later time using Scheduler. If this option is not selected, the file is not saved to Incentives storage. See the Scheduler Module section for more information on using Scheduler.

  12. Click Done.

Importing data from MySQL

From the Composer module, you can import data into tables from MySQL.

Important

Before you can import data from MySQL, ensure that you have Varicent ELT enabled on your model.

  1. From the Composer module, find the table that you want to import data into.

  2. Click Import.

  3. Click MySQL.

  4. To connect to a new MySQL account:

    1. Click New connection.

    2. Type a connector name.

    3. Enter your MySQL host, port, database, username, and password login details.

    4. Click Next.

    Important

    When you connect to a MySQL account, all users in the same Varicent ELT organization will be able to view and access that account.

  5. Select a MySQL account to connect to from the list of connections.

  6. Select the data you'd like to include or obfuscate and click Finish.

  7. Confirm the data sets and click Next.

  8. Map the columns in the target table, listed under Destination table columns, to the Source file fields in MySQL.

    To list only the unmatched columns, turn on the Show only unmatched columns toggle.

    Click Next.

  9. Select any or all of the following run options:

    Option

    Description

    Update existing rows

    Updates any existing rows of data with data that's contained in the imported rows.

    Run this import now

    Runs the import immediately. If you want to save the current import and process it later, clear this checkbox.

    Use incremental import

    Imports the table rows into the database in smaller batches instead of committing the entire import in one batch. Use this option to process database calls, such as report generation in the Sales Portal, during a large data import.

    Save this import as

    Saves the import so you can run it at a later time using Scheduler. If this option is not selected, the file is not saved to Incentives storage. See the Scheduler Module section for more information on using Scheduler.

  10. Click Done.

Importing data from SQL server

From the Composer module, you can import data into tables from SQL Server.

Important

Before you can import data from SQL Server, ensure that you have Varicent ELT enabled on your model.

  1. From the Composer module, find the table that you want to import data into.

  2. Click Import.

  3. Click SQL Server.

  4. To connect to a new SQL Server account:

    1. Click New connection.

    2. Type a connector name.

    3. Enter your SQL Server host, port, database, username, and password login details.

    4. Click Next.

    Important

    When you connect to a SQL Server account, all users in the same Varicent ELT organization will be able to view and access that account.

  5. Select a SQL Server account to connect to from the list of connections.

  6. Select a data set and click Next.

  7. Select the data you'd like to include or obfuscate and click Next.

  8. Confirm the data set and click Finish.

  9. Map the columns in the target table, listed under Destination table columns, to the Source file fields in SQL Server.

    To list only the unmatched columns, turn on the Show only unmatched columns toggle.

    Click Next.

  10. Select any or all of the following run options:

    Option

    Description

    Update existing rows

    Updates any existing rows of data with data that's contained in the imported rows.

    Run this import now

    Runs the import immediately. If you want to save the current import and process it later, clear this checkbox.

    Use incremental import

    Imports the table rows into the database in smaller batches instead of committing the entire import in one batch. Use this option to process database calls, such as report generation in the Sales Portal, during a large data import.

    Save this import as

    Saves the import so you can run it at a later time using Scheduler. If this option is not selected, the file is not saved to Incentives storage. See the Scheduler Module section for more information on using Scheduler.

  11. Click Done.

Importing data from Oracle

From the Composer module, you can import data into tables from Oracle.

Important

Before you can import data from Oracle, ensure that you have Varicent ELT enabled on your model.

  1. From the Composer module, find the table that you want to import data into.

  2. Click Import.

  3. Click Oracle.

  4. To connect to a new Oracle account:

    1. Click New connection.

    2. Type a connector name.

    3. Enter your Oracle host, port, sid, database, username, and password login details.

    4. Click Next.

    Important

    When you connect to an Oracle account, all users in the same Varicent ELT organization will be able to view and access that account.

  5. Select an Oracle account to connect to from the list of connections.

  6. Select the data you'd like to include or obfuscate and click Finish.

  7. Confirm the data sets and click Next.

  8. Map the columns in the target table, listed under Destination table columns, to the Source file fields in Oracle.

    To list only the unmatched columns, turn on the Show only unmatched columns toggle.

    Click Next.

  9. Select any or all of the following run options:

    Option

    Description

    Update existing rows

    Updates any existing rows of data with data that's contained in the imported rows.

    Run this import now

    Runs the import immediately. If you want to save the current import and process it later, clear this checkbox.

    Use incremental import

    Imports the table rows into the database in smaller batches instead of committing the entire import in one batch. Use this option to process database calls, such as report generation in the Sales Portal, during a large data import.

    Save this import as

    Saves the import so you can run it at a later time using Scheduler. If this option is not selected, the file is not saved to Incentives storage. See the Scheduler Module section for more information on using Scheduler.

  10. Click Done.

Importing data from Amazon Aurora

From the Composer module, you can import data into tables from Amazon Aurora.

Important

Before you can import data from Amazon Aurora, ensure that you have Varicent ELT enabled on your model.

  1. From the Composer module, find the table that you want to import data into.

  2. Click Import.

  3. Click Amazon Aurora.

  4. To connect to a new Amazon Aurora account:

    1. Click New connection.

    2. Type a connector name.

    3. Select a compatibility mode and enter your Amazon Aurora host, port, database, username, and password login details.

    4. Click Next.

    Important

    When you connect to an Amazon Aurora account, all users in the same Symon organization will be able to view and access that account.

  5. Select an Amazon Aurora account to connect to from the list of connections.

  6. Select the data you'd like to include or obfuscate and click Finish.

  7. Confirm the data sets and click Next.

  8. Map the columns in the target table, listed under Destination table columns, to the Source file fields in Amazon Aurora.

    To list only the unmatched columns, turn on the Show only unmatched columns toggle.

    Click Next.

  9. Select any or all of the following run options:

    Option

    Description

    Update existing rows

    Updates any existing rows of data with data that's contained in the imported rows.

    Run this import now

    Runs the import immediately. If you want to save the current import and process it later, clear this checkbox.

    Use incremental import

    Imports the table rows into the database in smaller batches instead of committing the entire import in one batch. Use this option to process database calls, such as report generation in the Sales Portal, during a large data import.

    Save this import as

    Saves the import so you can run it at a later time using Scheduler. If this option is not selected, the file is not saved to Incentives storage. See the Scheduler Module section for more information on using Scheduler.

  10. Click Done.

Importing data from Amazon S3

From the Composer module, you can import data into tables from Amazon S3.

Important

Before you can import data from Amazon S3, ensure that you have Varicent ELT enabled on your model.

  1. From the Composer module, find the table that you want to import data into.

  2. Click Import.

  3. Select Amazon S3.

  4. Select an Amazon AWS account to connect from the list of connections. To learn how to connect to a new Amazon AWS account, read: Connecting to an Amazon AWS account.

  5. Click Next.

  6. Select a data set and click Next.

  7. Select the data you'd like to include or obfuscate and click Next.

  8. Confirm the data set and click Finish.

  9. Map the columns in the target table, listed under Destination table columns, to the Source file fields in Amazon AWS.

    To list only the unmatched columns, turn on the Show only unmatched columns toggle.

    Click Next.

  10. Select any or all of the following run options:

    Option

    Description

    Update existing rows

    Updates any existing rows of data with data that's contained in the imported rows.

    Run this import now

    Runs the import immediately. If you want to save the current import and process it later, clear this checkbox.

    Use incremental import

    Imports the table rows into the database in smaller batches instead of committing the entire import in one batch. Use this option to process database calls, such as report generation in the Sales Portal, during a large data import.

    Save this import as

    Saves the import so you can run it at a later time using Scheduler. If this option is not selected, the file is not saved to Incentives storage. See the Scheduler Module section for more information on using Scheduler.

  11. Click Done.

Connecting to an Amazon AWS account

To use the Amazon S3 connector in Incentives, you must first configure the policy and role in your Amazon AWS account.

Before importing data into Incentives, ensure that you create an S3 bucket from your Amazon AWS account, and upload the files to import into that bucket.

Follow these steps to configure the policy and role in your Amazon AWS account:

  1. From the Composer module, follow the steps to begin a data import from Amazon S3.

  2. Click New connection.

  3. Type a connector name.

  4. Enter your account ID and bucket name, and click Next.

  5. Complete the following steps from your Amazon AWS account:

    1. Open IAM.

    2. Click PoliciesCreate policy.

    3. Copy the policy body shown in Incentives and paste it under the JSON tab.

    4. Click Next: Tags.

    5. Click Next: Review.

    6. Type a name and click Create policy.

    7. Click RolesCreate role.

    8. Select AWS account.

    9. Select Another AWS account. Copy the account ID from Incentives and paste it into the Account ID text box.

    10. Select Require external ID. Copy the external ID from Incentives and paste it into the External ID text box.

    11. Click Next.

    12. Select the policy you created earlier and click Next.

    13. Copy the role name generated in Incentives and paste it into the Role name text box.

    14. Click Create role.

  6. In Incentives, continue with the steps to configure your data import.

Importing data from Amazon Redshift

From the Composer module, you can import data into tables from Amazon Redshift.

Important

Before you can import data from Amazon Redshift, ensure that you have Varicent ELT enabled on your model.

  1. From the Composer module, find the table that you want to import data into.

  2. Click Import.

  3. Click Amazon Redshift.

  4. To connect to a new Amazon Redshift account:

    1. Click New connection.

    2. Type a connector name, host, port, database, username, and password for your account.

    3. Click Next.

  5. Select an Amazon Redshift account to connect to from the list of connections.

  6. Select a data set and click Next.

  7. Select the data you'd like to include or obfuscate and click Next.

  8. Confirm the data set and click Finish.

  9. Map the columns in the target table, listed under Destination table columns, to the Source file fields in Amazon Redshift.

    To list only the unmatched columns, turn on the Show only unmatched columns toggle.

    Click Next.

  10. Select any or all of the following run options:

    Option

    Description

    Update existing rows

    Updates any existing rows of data with data that's contained in the imported rows.

    Run this import now

    Runs the import immediately. If you want to save the current import and process it later, clear this checkbox.

    Use incremental import

    Imports the table rows into the database in smaller batches instead of committing the entire import in one batch. Use this option to process database calls, such as report generation in the Sales Portal, during a large data import.

    Save this import as

    Saves the import so you can run it at a later time using Scheduler. If this option is not selected, the file is not saved to Incentives storage. See the Scheduler Module section for more information on using Scheduler.

  11. Click Done.

Importing data from Hubspot

From the Composer module, you can import data into tables from Hubspot.

Important

Before you can import data from Hubspot, ensure that you have Varicent ELT enabled on your model.

  1. From the Composer module, find the table that you want to import data into.

  2. Click Import.

  3. Click Hubspot.

  4. To connect to a new Hubspot account:

    1. Click New connection.

    2. Type a connector name and click Next.

    3. Enter your the email address and password for your Hubspot account.

    4. Click Log in.

    5. Select the Hubspot account you'd like to connect to and click Choose Account.

    6. Click Connect app.

  5. Select a Hubspot account to connect to from the list of connections.

  6. Select a start date and click Next.

  7. Select a data set and click Next.

  8. Select the data you'd like to include or obfuscate and click Next.

  9. Confirm the data set and click Finish.

  10. Map the columns in the target table, listed under Destination table columns, to the Source file fields in Hubspot.

    To list only the unmatched columns, turn on the Show only unmatched columns toggle.

    Click Next.

  11. Select any or all of the following run options:

    Option

    Description

    Update existing rows

    Updates any existing rows of data with data that's contained in the imported rows.

    Run this import now

    Runs the import immediately. If you want to save the current import and process it later, clear this checkbox.

    Use incremental import

    Imports the table rows into the database in smaller batches instead of committing the entire import in one batch. Use this option to process database calls, such as report generation in the Sales Portal, during a large data import.

    Save this import as

    Saves the import so you can run it at a later time using Scheduler. If this option is not selected, the file is not saved to Incentives storage. See the Scheduler Module section for more information on using Scheduler.

  12. Click Done.

Importing data from Snowflake

From the Composer module, you can import data into tables from Snowflake.

Important

Before you can import data from Snowflake, ensure that you have Varicent ELT enabled on your model.

  1. From the Composer module, find the table that you want to import data into.

  2. Click Import.

  3. Click Snowflake.

  4. To connect to a new Snowflake account:

    1. Click New connection.

    2. Type a connector name.

    3. Enter your Snowflake URL, role, warehouse, database name, username, and password login details.

    4. Click Next.

    Important

    When you connect to a Snowflake account, all users in the same Varicent ELT organization will be able to view and access that account.

  5. Select a Snowflake account to connect to from the list of connections.

  6. From the Directory drop-down list, select a file to import data from and click Next.

  7. Select the data you'd like to include or obfuscate and click Next.

  8. Confirm the data set and click Finish.

  9. Map the columns in the target table, listed under Destination table columns, to the Source file fields in Snowflake.

    To list only the unmatched columns, turn on the Show only unmatched columns toggle.

    Click Next.

  10. Select any or all of the following run options:

    Option

    Description

    Update existing rows

    Updates any existing rows of data with data that's contained in the imported rows.

    Run this import now

    Runs the import immediately. If you want to save the current import and process it later, clear this checkbox.

    Use incremental import

    Imports the table rows into the database in smaller batches instead of committing the entire import in one batch. Use this option to process database calls, such as report generation in the Sales Portal, during a large data import.

    Save this import as

    Saves the import so you can run it at a later time using Scheduler. If this option is not selected, the file is not saved to Incentives storage. See the Scheduler Module section for more information on using Scheduler.

  11. Click Done.

Importing data from Dynamics

From the Composer module, you can import data into tables from Microsoft Dynamics.

Important

Before you can import data from Dynamics, ensure that you have Varicent ELT enabled on your model.

  1. From the Composer module, find the table that you want to import data into.

  2. Click Import.

  3. Click Dynamics.

  4. To connect to a new Dynamics account:

    1. Click New connection.

    2. Type a connector name.

    3. Enter your Dynamics URL.

    4. Click Next.

    5. To grant Incentives access to your Dynamics account, click Accept.

    Important

    When you connect to a Dynamics account, all users in the same Varicent ELT organization will be able to view and access that account.

  5. Select a Dynamics account to connect to from the list of connections.

  6. Select a data set and start date. Click Next.

  7. Select the data you'd like to include or obfuscate and click Next.

  8. Confirm the data set and click Finish.

  9. Map the columns in the target table, listed under Destination table columns, to the Source file fields in Dynamics.

    To list only the unmatched columns, turn on the Show only unmatched columns toggle.

    Click Next.

  10. Select any or all of the following run options:

    Option

    Description

    Update existing rows

    Updates any existing rows of data with data that's contained in the imported rows.

    Run this import now

    Runs the import immediately. If you want to save the current import and process it later, clear this checkbox.

    Use incremental import

    Imports the table rows into the database in smaller batches instead of committing the entire import in one batch. Use this option to process database calls, such as report generation in the Sales Portal, during a large data import.

    Save this import as

    Saves the import so you can run it at a later time using Scheduler. If this option is not selected, the file is not saved to Incentives storage. See the Scheduler Module section for more information on using Scheduler.

  11. Click Done.

Importing data from Workday

From the Composer module, you can import data into tables from Workday.

Important

Before you can import data from Workday, ensure that you have Varicent ELT enabled on your model.

  1. From the Composer module, find the table that you want to import data into.

  2. Click Import.

  3. Click Workday.

  4. To connect to a new Workday account:

    1. Click New connection.

    2. Type a connector name, domain, tenant, username, and password for your account.

    3. Select a country code.

    4. Click the Exclude Inactive Workers or Exclude Contingent Workers checkbox if needed.

    5. Click Next.

  5. Select a Workday account to connect to from the list of connections.

  6. Select the data you'd like to include or obfuscate and click Next.

    Important

    You can only select 500 columns to include or obfuscate. To choose the first 500 columns, select the First 500 checkbox. To select different columns, deselect the columns you don't want, and click the checkboxes of the columns you want to include or obfuscate.

  7. Click Finish.

  8. Map the columns in the target table, listed under Destination table columns, to the Source file fields in Workday.

    To list only the unmatched columns, turn on the Show only unmatched columns toggle.

    Click Next.

  9. Select any or all of the following run options:

    Option

    Description

    Update existing rows

    Updates any existing rows of data with data that's contained in the imported rows.

    Run this import now

    Runs the import immediately. If you want to save the current import and process it later, clear this checkbox.

    Use incremental import

    Imports the table rows into the database in smaller batches instead of committing the entire import in one batch. Use this option to process database calls, such as report generation in the Sales Portal, during a large data import.

    Save this import as

    Saves the import so you can run it at a later time using Scheduler. If this option is not selected, the file is not saved to Incentives storage. See the Scheduler Module section for more information on using Scheduler.

  10. Click Done.

Adding new fields

When importing data from a file in the Composer module, you can create new fields from existing columns. From these new fields, you can also create subitems, which are a segment of the data in an existing column.

  1. From the Composer module, find the table that you want to import data into.

  2. Click Import.

  3. Select a file type to import data from.

  4. Configure the data import.

  5. When mapping the import file columns to the table columns, click Manage fields.

  6. Click Add new field.

  7. Select a source field from the drop-down list and type a name for the new field.

  8. To add a subitem, select one of the following options:

    Option

    Description

    Create a subitem by selecting characters to include

    Select this option to type the number of characters in the field to include. For example, type 1-5 if you want to include the first five characters of the field.

    Use a delimiter to create a subitem for this field

    Select this option to type a delimiter and select the index to create a subitem. For example, if you have a Name field that contains first and last names separated by a space (for example, Dan Huddle) and you want to add a First Name field, you would type a space as the delimiter and select an Index of 1 since the first name is the first item before the delimiter.

  9. Click Add.

Merging fields

When importing data from a file in the Composer module, you can combine two columns in your data source into one column. For example, if you have a month and a year field, you can merge the two and create one date field.

  1. From the Composer module, find the table that you want to import data into.

  2. Click Import.

  3. Select a file type to import data from.

  4. Configure the data import.

  5. When mapping the import file columns to the table columns, click Manage fields.

  6. Click Merge Fields.

  7. Type a name for the new field.

  8. Select the fields you want to merge from the Available fields for merging drop-down list.

  9. Select the delimiter from the Delimiter drop-down list.

    When importing from a text file, you can choose a special character to import fields that contain a delimiter. You can choose from these delimiters: comma, semicolon, tab, pipe (|), and period. This is useful if you have data that contains a comma but is importing from a comma delimited file. Without some sort of special character, there would be no way to import a field containing a comma from that kind of file.

  10. Click Add.

Using effective dates in table imports

When you import data into tables with effective dates in Varicent Incentives, you must map the effective date columns to the appropriate date columns in the import.

Optionally, you can click Defaults to select a default effective start date and a default effective end date to use for all imported records.

The effective start date is a key column so it must be mapped to a date field in the import file or set to a default date. The values cannot be empty or null.

The effective end date is not a key column so it does not have to be mapped. If the effective end date is unmapped, imported records have a date of 12/31/9998.

Imported effective start and effective end values must follow the rules of effective dating. That is, the dates must be between 1/1/1753 and 12/31/9998 and the effective start date must be the same day or earlier than the effective end date. The import file cannot contain multiple records with matching keys and overlapping effective dates.

The effective start date and effective end date mapping can be modified when editing a saved import.

Editing EffectiveDate tables using TSAPI

Our TSAPI allows for the insertion and manipulation of EffectiveDate tables. All payees have the ability to edit EffectiveDate tables using the TSAPI, without the need to do so through web forms.

You can use POST/PATCH via TSAPI to do the following:

POST

  • Add new rows with or without effective end date

  • Add rows with effective start date ahead of the existing rows

  • Overwrite existing rows

PATCH

  • Update effective end date

  • Update non-key columns

Refreshing table data

In Varicent Incentives, in Composer, you can refresh the table after importing data.

  1. In Composer, click the table.

  2. In the more options menu that displays on the right, click Refresh.

    table_refresh.jpg

Troubleshooting data imports

If a data import is slow, you can try using an alternate import method to improve database imports.

Symptoms

Import is slow compared to a previous import of the same data.

Resolving the problem

You can use an alternate import method for database imports. This method can improve import times, but should only be used if an import is significantly slow.

When you run a database import, select Path 1 as the import method.

Note

You cannot use this method if you want to use a text driver to import your data.

  1. In Composer, click the table.

  2. Click Import.

  3. Select Import data from Varicent ICM storage.

  4. Select Database Import.

  5.  

  6. From the Import Method menu, select Path 1.