Data Loader

The Data Loader is a powerful add-on to import data in bulk.

The Data Loader is an add-on for eBECAS/EDMISS for bulk import of data. It can be used to insert or update records. 

When importing data, the Data Loader reads, extracts, validates, and loads data from a comma-separated values (CSV) or Excel (XLS | XLSX) file.

About the Data Loader Configuration Files

• The Data Loader requires a configuration file for each dataset (source file). These files are created by Equator IT and provided to you.

• A configuration file is specific to each dataset. Any changes to the format or structure of the dataset might require an update to the configuration file.

• All configuration files have the extension .eit

Using the Data Loader

Setting Up the Import

Before you can use the Data Loader, you need to set up the import. You need to do it only once for a specific dataset. However, the configuration is only available to the user and on the device that was used to set it up. If you or other users need to import the same dataset on a different device, you/they need to go through this set up process.

1. Open the Data Loader by going to Utilities > Import > Data Loader.

2. Check if you already have the configuration for the dataset on the ‘Available Imports’. If you have it, continue to the next section.

3. If you do not have it, click on ‘Import New’ and then browse and select the configuration file provided by Equator IT.

4. After selecting the file, the configuration will be available on the ‘Available Imports’ section.

Preparing the Dataset to Import

After you set up the import (see section Setting up the Import), you need to prepare the source file with the data to import.

1. From the Data Loader window, click on the configuration that you want to use on the ‘Available Imports’ section.

2. If this is the first time you are preparing the dataset, review the section ‘Columns Definition’ to explore all the details of the columns that the Data Loaders is expecting on the source file. The name of the columns on the file must match exactly the names of the columns displayed in this section. Each row in the ‘Columns Definition’ defines each column on the dataset.

3. If this is the first time you are using this import setup, you can click on ‘Create Import Template’ to generate and download a template in Excel with the columns required for the import. If you would like to include specific details for each column, select the option ‘Include Column Details’ before creating the import template.

   

Important Notes for Import Templates

When entering data to an import template, please note that:

1. The templates have up to 3 rows: 

   1. Header: This is the name of the column. Do NOT modify them.

   2. Column Description (If Include Column Details option is selected when Create Import Template): This has the details of the column and the values that it requires. Please make sure to follow the specified data types and maximum letter lengths (if text type) as stated. Remove this row before importing the template to eBECAS/EDMISS. 

   3. Column options (If Include Column Details option is selected when Create Import Template): For details for the columns that have a restricted value, the options available will be on this row. Remove this row before importing the template to eBECAS/EDMISS.

2. There are four types of data fields for import templates.

   • Date: Use one of the following formats exclusively and do not use multiple formats: 

     • dd/mm/yyyy

     • dd-mmm-yyyy

     • mm/dd/yyyy

     • mmm-dd-yyyy

   • Number: Enter only positive integers, excluding negative values. Do not include decimal places. 

   • Decimal: Enter a number excluding negative values, including decimal places. 

   • Text: Enter texts within the maximum allowable character limit written next of it. 

3. Do not leave empty lines at the bottom of templates. 

4. Imports directly transfer the entered values into the system without validating them for accuracy, such as calculation correctness or typos. It’s essential to verify data before entering it, as the system will input values exactly as they appear in the import template.

5. For text fields that allow free input rather than must be from predefined values in the system(usually described in the Description), a blank value in the import template will also input a blank value in the system.

Importing the File

The import process has 4 steps:

Step 1: Choosing the Dataset (source file)

1. From the Data Loader Window, click on the ‘Import File’ tab at the top. If this tab is not enabled, you need to select the import configuration first from the ‘Loader Config’ tab.

2. Select the file you want to import. The extensions allowed are: .csv, .xls, .xlsx.

3. If the file has multiple sheets, enter the name of the sheet that has the data to import. You can also choose ‘Use First Sheet’, for the Data Loader to use the first sheet in the file.

4. Select the row where the headers are (if not in the first row).

5. If your dataset has dates, select the format for the dates.

Step 2: Previewing the Dataset

1. From the ‘Import’ panel, click on ‘Preview File’.

2. The data of the file will display in the ‘File Preview’ section.

3. The number of rows found will display under the ‘File Preview’ section.

4. If it does not look right, update the file accordingly and preview the file again until you are happy with the format of the data.

Step 3: Validating the Dataset (Highly Recommended)

Before running the import, it is important that you validate your dataset to make sure there will not be any unexpected errors. The validation will run the same process as the import, but it will not update or create any data.

1. From the ‘Import’ panel, click on ‘Validate Import’.

2. A pop-up window will display the results of the validation.

3. If there are no errors, you can continue to the next section. However, if there are any errors, click on 'OK'. Then, you will have the option to save the file with the description of the errors.

4. Open the file with the errors and fix the related records on the source file or in eBECAS/EDMISS.

5. Once the errors are fixed, repeat the process until there are no errors in the dataset.

Step 4: Running the Import

After validating the file, you are ready to run the import. This step will modify the data in eBECAS/EDMISS so make sure you have no errors before running it (see section Validating the dataset).

1. From the ‘Import’ panel, click on ‘Run Import’.

2. A confirmation window will display with the result of the validation. This is the last opportunity to fix the errors (if any).

3. If you would like to make changes before committing the changes, click on ‘Cancel’. This will abort the import process and no data will be committed. To see the errors, you have the option to save the 'Import Results' to a file. If you need to fix any errors in the dataset or in eBECAS/EDMISS, do so and start the process again.

4. If you are happy with the summary and want to go ahead with the import and commit the changes, click on ‘Ok’.

5. At this point, you will have the option to store the results in a file so you can review the results for each row. Choose the name and location of the file and open it to see the results.