#2257: ImportStorage.kt

projectforge-rest/src/main/kotlin/org/projectforge/rest/importer/ImportStorage.kt Abstract entity — REST Importer Layer, projectforge-rest/src/main/kotlin/org/projectforge/rest/importer/ImportStorage.kt 240 lines · 127 code · 79 comments · 34 blank
Abstract base class for all import storage implementations. Manages import reconciliation between uploaded data and database entities, column mapping/detection, pair entry lifecycle with error/warning tracking, and display filtering for import UIs. Parameterized by the modified entity type O (extending ImportPairEntry.Modified<O>).

Architecture

Class Hierarchy

ImportStorage<O : ImportPairEntry.Modified<O>> is the core abstract class in the import subsystem. Concrete importers extend this to define how uploaded data maps to domain entities. It operates alongside ImportSettings, ImportPairEntry, AbstractImportJob, and AbstractImportPageRest.

Key Components

ComponentTypePurpose
importSettingsConstructor paramImmutable import configuration (delimiter, encoding, etc.)
pairEntriesMutableList<ImportPairEntry<O>>All import entries pairing read (uploaded) and stored (database) entities
detectedColumnsMutableMap<String, ImportFieldSettings>Columns detected from the uploaded file headers
unknownColumnsMutableList<String>Columns found in the file but not mapped
columnMappingMutableMap<Int, ImportFieldSettings>Position-to-field mapping for parsing
errors / warningsThread-safe listsGlobal error and warning messages with synchronized access
displayOptionsDisplayOptions (inner class)Toggle visibility of new/modified/unmodified/imported/deleted/faulty/unknown entries
targetEntity / targetEntityTitleOptional fieldsFor multi-entity imports, identifies the current target and its display title

Reconciliation Flow

The core reconciliation mechanism is a template method pattern:

  1. reconcileImportStorage(rereadDatabaseEntries) — final method, sets hasBeenReconciled = true, calls hook, marks all entries reconciled
  2. doReconcileImportStorage(rereadDatabaseEntries) — protected open hook for subclass implementation (default no-op)
  3. After reconciliation, pairEntries show status per entry: NEW, MODIFIED, UNMODIFIED, IMPORTED, DELETED, FAULTY

Entry Lifecycle

MethodPurpose
prepareEntity(): OAbstract — creates a new blank entity for population
prepareImportPairEntry(): ImportPairEntry<O>Creates a pair entry with a prepared entity, ready for parsing with error tracking
addEntry(entry)Adds to pairEntries with auto-incrementing ID
setProperty(obj, fieldSettings, value): BooleanOpen — custom property setting; returns true if handled, false for auto-setting
commitEntity(obj: O)Abstract — stores/skips the entity after all properties are set
commitEntity(pairEntry)Open — preferred commit method using pair entry (calls commitEntity with read entity)
createEntries(displayOptions)Converts pair entries to display entries, applying display filter and transferring errors

Thread Safety

The errors and warnings lists are protected by synchronized blocks. Public read access returns defensive copies. The pairEntries list is also synchronized during ID assignment.

Info Export

ImportStorageInfo is generated on-the-fly via the info property, providing a snapshot of counts (new, modified, unmodified, imported, deleted, faulty, unknown) for UI rendering.

Subclasses (on this branch)

Git History

868d6abb7 2025 -> 2026
76f216b0f AbstractPagesRest: param request added to open functions, WIP. address vcf upload
16ab7c6ef AbstractImportPage fix: Long - Integer comparison
45479d226 WIP: Import of creditor invoices
8717a58b8 WIP: Import of creditor invoices
f4540acfa WIP: Import of creditor invoices
fdd683ca7 WIP: Import of creditor invoices
c9dff4cb8 WIP: Import of creditor invoices
ffc71ab7e WIP: Import of creditor invoices
95363546e WIP: Import of creditor invoices.
f9f42dae1 WIP: ImportStorage
63081666f Source file headers: 2024-> 2025.
61f05ce90 Migration stuff in progress...
b6092df09 Copyright 2023 -> 2024
ab45d51fa Copyright 2001-2022 -> 2001-2023.
1ea8d4062 WIP: Jobs and Bank account records.
18cf43df0 WIP: Jobs and Bank account records.
163492453 WIP: Jobs and Bank account records.
09b3f0df7 WIP: Jobmonitor. ListStatisticsSupport -> MarkdownBuilder
a1646e91a WIP: Banking plugin (and new import module).
9ba1738ea WIP: Banking plugin (and new import module).
c3331c88e WIP: Banking plugin (and new import module). DynamicDropArea: copy&paste bug fixed (function renamed)
1b6b960a9 WIP: Banking plugin (and new import module).
72a042463 WIP: Banking plugin (and new import module).
6ec22a30d WIP: Banking plugin (and new import module).
d38c01233 WIP: Banking plugin (and new import module).
67333b119 WIP: Banking plugin (and new import module).
750e60742 WIP: Banking plugin (and new import module).
0936be3ed WIP: Banking plugin (and new import module).
03796e3c6 WIP: Banking plugin (and new import module).
e118c101d WIP: Banking plugin (and new import module).
9ccbfdfc2 WIP: Banking plugin (and new import module).
5f7ef41b8 Copyright 2021 -> 2022
ceb63e8a1 Source code header: (C) 2001-2021.
7c79f1922 Copyright of source header -> 2020.
45cad02e9 IHK-Plugin added ( Will replace "Ihk-Export"-Plugin in the long run )
dd5ca38ac CopyRight of all java file-header updated or created.
9ebb88522 Initial commit