Purpose: This article explains how Administrate processes CSV imports behind the scenes—what happens before any data changes, how matching decisions are made during processing, why some rows fail while others succeed, and how to run imports safely at scale.
Related: Importing data into Administrate
Contents
- Mental model: imports run in two phases
- Phase 1: Pre-processing (Validation phase)
- Field matching and mapping behavior
- Phase 2: Execution (Processing phase)
- Matching and identity resolution
- Validation vs processing failures
- Order of operations and dependencies
- Performance and scale considerations
- Safe import strategy
- Re-imports, corrections, and recovery strategy
- Next steps
Mental model: imports run in two phases
Administrate imports are queued jobs. Once you start an import, it enters a processing queue and is executed as a background task. During execution, the system evaluates each row, determines whether it should create or update a record, applies changes, and logs outcomes.
Import behavior is easiest to understand as two phases:
- Pre-processing (Validation phase) — confirms the file and mapping are usable before processing begins.
- Execution (Processing phase) — evaluates rows, performs matching, applies changes, and records row-level results.
Phase 1: Pre-processing (Validation phase)
This phase is about catching problems before the system attempts to change data. If validation fails, the import can be blocked early (meaning nothing is applied).
What validation typically checks
- File structure: correct delimiter/CSV shape, headers present where expected, consistent column counts.
- Required fields: required columns are present and mapped.
- Mapping integrity: fields are mapped to compatible Administrate fields (e.g., date fields mapped to date columns).
- Basic formatting: obvious type issues (e.g., invalid dates) may be flagged before processing.
Field matching and mapping behavior
During import processing, Administrate attempts to automatically match incoming CSV columns to supported system fields.
If automatic matching cannot confidently determine the correct destination field, administrators must manually map CSV columns before the import can proceed.
Required fields must always be mapped before records can be validated and processed successfully.
Field mapping validation may also fail when:
- Required fields are missing from the source file
- CSV headers do not match expected field formats
- Mapped fields contain incompatible data types
- Relationship or lookup fields cannot resolve valid records
Administrate validates field mapping before records are committed to the import process.
Where to go next:
Phase 2: Execution (Processing phase)
Once validation passes, the job runs in the queue. During processing, Administrate evaluates rows and attempts to apply changes.
Row-by-row behavior
Each row is processed independently. For each row, the system:
- Reads values from the CSV row.
- Determines identity by applying matching rules (to find an existing record or decide to create a new one).
- Applies changes to mapped fields (create/update).
- Records a row result (success, warning, or error) for traceability.
Partial success is normal
Yes—partial success is expected. Some rows can succeed while others fail. A failed row does not automatically block all other rows from being applied.
This is why verification is essential after any import run—especially for large files or updates.
Where to go next:
Matching and identity resolution (in processing context)
Matching determines whether each row will create a new record or update an existing one. In practice, most “surprising” import outcomes come from identity resolution.
Important: This article explains how matching behaves during processing. The canonical rules (what Administrate treats as identifiers, how lookups work, and how safe updates should be performed) are maintained in these system-level references:
- Imports & Data Logic: Matching, Updates, and Safe Operation
- Identifiers, Lookups, and Matching Rules
Common matching outcomes you’ll see during imports
- Unexpected duplicates created: matching keys were missing/incorrect, or values didn’t match existing records as expected.
- Rows fail due to “not found” references: a lookup field pointed to a value that doesn’t exist (or isn’t visible in the current context).
- Records updated incorrectly: identifiers matched a different record than intended.
Validation vs processing failures (and what they mean)
Failures tend to happen in two distinct places:
A) Blocked before processing starts
These are usually file/mapping/format issues that prevent the queued job from applying any changes.
- Missing required columns
- Unmapped required fields
- Incompatible mappings (wrong data type)
- File formatting issues
B) Row-level processing errors
These occur while the job is running. Some rows are applied successfully; others fail and are logged.
- Invalid data types at runtime (bad date formats, invalid numeric values)
- Lookup/reference values missing (e.g., referencing a related record that doesn’t exist)
- Constraint or dependency issues (row requires a relationship that cannot be resolved)
- Conflicting values (row attempts to set a value that violates rules/constraints)
Where to troubleshoot:
Order of operations and dependencies
Imports become more fragile as relationships and dependencies increase. When you’re importing data that depends on other data (or references other objects), ordering matters.
Common dependency patterns
- Lookup fields: if a row references another object by name/code/identifier, that referenced value typically must already exist.
- Relationship-driven fields: some fields are only valid when related objects exist or are in a compatible state.
- Visibility boundaries: if imports depend on records that are not visible/accessible under your current context, lookups may fail.
Recommended approach:
- Import “foundation” objects first (the entities other imports will reference).
- Then import dependent objects that reference those foundations.
- Verify results after each major step before proceeding.
Related conceptual references:
- How Administrate Works: Core Concepts & Relationships
- Accounts, Contacts, Companies, and Visibility Boundaries
Performance and scale considerations
Because imports are queued jobs, large imports can take time. The safest strategy for large updates is deliberate, staged execution.
Safe scale patterns
- Start small: run a pilot import with a subset of rows.
- Verify early: confirm matching behavior and results before scaling up.
- Prefer incremental updates: avoid “mega imports” that make rollback/reconciliation painful.
- Track what changed: keep a copy of the CSV used and a timestamped record of the run.
Safe import strategy (recommended)
Imports can permanently create or update records. Before running large imports—especially updates—use this checklist:
- Confirm identifiers/matching: understand what fields determine identity and updates.
- Run a pilot: import a small subset and validate outcomes.
- Verify results: reconcile imported data immediately after the run.
- Scale deliberately: proceed in batches rather than one large run.
- Keep the source: save the exact CSV used for traceability.
Re-imports, corrections, and recovery strategy
Because imports can partially succeed and do not automatically roll back successful rows, it is critical to understand how to safely correct issues after an import has run.
If duplicates were created
Duplicates usually occur when matching identifiers were missing, incorrect, or did not match existing records exactly.
Immediate steps:
- Stop further imports until matching logic is confirmed.
- Identify the duplication pattern (which identifier failed?).
- Use filtering or reporting to isolate affected records.
- Determine whether records should be merged, manually corrected, or updated via a corrective import.
Prevention going forward:
- Confirm identifier fields before running large updates.
- Run a small pilot import to verify matching behavior.
If some rows failed (partial success scenario)
Since partial success is expected behavior, some rows may succeed while others fail.
Recommended correction workflow:
- Export or review the error report to identify failed rows.
- Correct only the failing rows in your CSV (do not re-run the full file blindly).
- Re-import only the corrected subset.
- Verify results immediately.
This controlled approach prevents accidental duplicate processing of already-successful rows.
What happens if you re-run the same file?
Re-running a file depends entirely on matching behavior:
- If identifiers correctly match existing records, the system will update those records.
- If identifiers are missing or mismatched, the system may create duplicates.
Before re-running any import:
- Confirm which field(s) determine identity.
- Confirm that identifier values are stable and accurate.
- Test with a small subset if unsure.
There is no automatic rollback
Administrate does not automatically reverse successful rows if other rows fail. Once a row is successfully processed, the change is applied.
This means:
- Imports should be approached deliberately.
- Large corrective updates should be staged.
- Verification is not optional for production data.
Safe correction pattern (recommended)
If an import produces unexpected results:
- Pause further imports.
- Assess scope of impact.
- Determine whether manual correction or targeted corrective import is appropriate.
- Run a small corrective test.
- Verify before scaling.
Understanding how matching, processing, and partial success work together is the key to running imports safely at scale.