Data Import

What would you like to do?

If the data is exported from an outside source, we strongly recommend you review the host_ids of the Core users to be updated to confirm the records being updated match an existing user.

  • If the host_id in the data import file doesn't match an existing user, a new, duplicate, user will be created.

  • Those duplicates will then need to be merged with the existing user (using the Potential Duplicate task).

  • Below are some proactive steps we recommend you take to minimize the introduction of duplicates.

  1. Run the advanced list template for "User Hostid". This list will display all existing host_IDs for a user, including any secondary host_ids. If either the host_id or secondary host_id match the user in your data import file, that user will be found and updated.

    1. From Core, go to Reporting , Manage Lists, Manage basic and advanced lists.

    2. Select List Templates and in the Template Category, select "All Categories."

    3. Locate the "User HostID" list.

    4. On the far right-hand side, click Run. The list is generated and displays.

  2. If there are users in Core whose host_id doesn't match an existing user in the data import file, prior to including them in a data import update file, run a preliminary data import to assign a host_id to these users. Be aware, however, that there are no checks & balances in place to identify host_ids already assigned to other users. Again, the advanced list (item #1 above) can assist in identifying if the host_id is already in use.

    1. To replace an existing user's current host_id with a new one or to add a host_id if they don't have one, use Data Import , User Management , Host ID.

    2. To assign a secondary host_id to an existing user and not replace the existing host_id, use: Data Import , Contact Card-Import/Refresh , Add Secondary Host ID.

You may also want to consider scrubbing your data before importing updates or building a database.

  1. From Core, select System tools and Data import.

  2. Select Create import job

  3. Select a Category

  4. Select an Import type

  5. In the File layout help section, select View file layout help and review the options available:

    • The screen displays the columns that should exist in your .CSV file. Bold items designate required fields.

    • Select Sample file to access sample Excel files you can download and use as a foundation for adding your data.

    • Select Detailed view to review the supported file layout before creating your import file. This view also identifies removeable data with a Yes in the Ability to Remove Data column. To remove data, in the .csv file, enter an upward ^ symbol (also known as a caret, hat, control, up, chevron, or wedge) in the field you want removed.

  1. From Core, select System tools and Data import.

  2. Select Create import job

  3. Select a Category; this choice is dependent on the type of data you wish to upload, and influences the area where this data will be located (ex. select Academic Groups for Course data)

  4. For Import type , verify that the correct type of data is selected. Options in this field are based on the selected Category.

  5. For Name , enter an alpha-numeric name that identifies the purpose of the data import.

  6. From Import process , select either:

    • Insert where not exists:Data that exists within the uploaded .CSV document, and does not exist in the system, is added automatically to the system. For example, if your school does not have English 101 as a course entered into the system, but it is contained as a value in the uploaded .CSV document, then it will be added into the system.

    • Differential update: Matching data in the system is overwritten with the data from the .CSV document.

      Note: Differential update is only available for enrollment-based Import types. It clears out all enrollment records for rosters in the section and then adds records back in based on what is in the file.

    • Insert only: Data from the .csv is added into the system; if an entry is present in both the system and in the .CSV document, then the new data from the .CSV is added as a new record.

  1. Select File layout help to review how your .csv data file should be structured. Review Prepare a data import file, above, to learn more about formatting specifics.

  2. Select one of these options:

    • Select Validate only before processing the file. The validation process checks to make sure all column headers are correct, all data is in the correct format and any fields needing to be in the database prior to the import are there.

    • Select Process now after you have received a clean, validated file. This process imports records into the database.

  1. From Core, select System tools and Data import.

  2. Select the context menu next to the appropriate import. Statuses include:

    • Created: The job has been successfully created.

    • Approved short queue: This status appears when a job is added for processing but has not yet been processed. This job should only take a few minutes to be processed as it contains less than 5,000 rows.

    • Approved long queue: This status appears when a job is added for processing but has not yet been processed. This job will process overnight as it contains between 5,000 and 10,000 rows.

    • Approved short queue: validate only: This status appears when a job is added for validation, contained a file with less than 5,000 rows and has passed validation. In order to process this file, a new job needs to be added.

    • Approved long queue: validate only: This status appears when a job is added for validation, contained a file with between 5,000 and 10,000 rows and has passed. In order to process this file, a new job needs to be added.

    • Failed validation: This status appears when a job is added for processing and there were problems with the file and no data was imported during this job. Clicking on validated file will display the error message row by row.
      Errors should be addressed and a new job added to process the file.

    • Failed validation: validate only: This status appears when a job is added for validation and there were problems with the file. Clicking on the validated file will display the error message row by row. Errors should be addressed and a new job added to re-validate the file.

    • Completed: The final status of a job added for processing, either for the short or long queue. Data from the file has been imported and you can now see it.

    • Completed with exceptions: The import was partially successful; items that could not be imported are identified and need to be corrected before uploading the import file again.

    Import exceptions can be reviewed if an import file contains data that could not be added into the system due to an error. For example, if a student id in the .csv file cannot be matched to an existing record in the database, then an exception is recorded.

    Select Download updated test score file with only exceptions to download a .csv file to your computer that contains only the erroneous data. Correct the data identified in the Information that couldn't be imported and reason columns, and then upload the .csv file again.

    Note: The updated import preview page featuring exception tracking is only available to test scores data imports currently.