Migrating from gINT: Best Practices and AlternativesgINT (by Bentley Systems) has been a staple in geotechnical and subsurface data management for decades. It excels at creating standardized borehole logs, laboratory data reports, and formatted subsurface deliverables. But organizations sometimes outgrow gINT due to changes in workflows, needs for cloud collaboration, licensing costs, or the desire to modernize data pipelines. This article covers practical best practices for migrating from gINT, common pitfalls to avoid, and alternative tools and approaches you can consider — both commercial and open source.
Why migrate from gINT?
There are several reasons teams decide to move away from gINT:
- Need for cloud-native collaboration and multi-user access without desktop-only licensing.
- Integration with modern GIS and BIM workflows (e.g., real‑time linking with ArcGIS, Revit, or other asset systems).
- Desire for centralized data management and an auditable single source of truth rather than dispersed project files.
- Automation and scripting flexibility using modern APIs and data-exchange formats (REST, JSON, SQL).
- Cost or support concerns, especially for organizations with shrinking geotechnical teams or those standardizing on other ecosystems.
Preparation: project planning and stakeholder alignment
Successful migration starts well before any files are converted.
- Identify stakeholders: geotechnical engineers, lab technicians, GIS/BIM teams, IT, records/archiving, procurement, and external consultants.
- Define success criteria: What counts as “complete”? Examples: all borehole logs readable, lab data retained and searchable, reporting templates reproduced to ±X% fidelity, new workflows live within Y months.
- Inventory existing assets: gINT project files, templates, custom macros/visual basic scripts, referenced images, lab databases, Excel exports, and any linked GIS or CAD files.
- Audit data quality: find missing metadata, inconsistent units, duplicate boreholes, corrupt files. Classify items as “clean”, “needs cleaning”, or “cannot be recovered”.
- Decide the migration scope: full historical migration vs. move-forward strategy (migrate active projects only, archive older projects as PDFs/archives).
Data model & export strategy
gINT data is typically stored in project files (.gdf/.gpr etc.) and custom database structures. A robust export strategy preserves both content and semantics.
- Export tabular data to neutral formats: CSV, Excel, or preferably relational exports (SQL dumps) that preserve table relationships and primary keys.
- Export logs and templates as standardized documents (PDF) to preserve presentation while extracting raw data separately.
- Capture metadata: project IDs, borehole IDs, coordinates (including datum and coordinate reference system), survey dates, lab test IDs, and units.
- Export images and attachments as separate files with references (e.g., image filenames linked to borehole IDs).
- If gINT connects to an external database (e.g., Access, SQL Server), capture the schema and relationships directly from the database rather than from exported reports.
Recommended exports:
- Raw data: CSV/Excel per table.
- Relational backup: SQL dump if possible.
- Documents: PDF copies of existing deliverables for compliance/record.
- Scripts/macros: Save any custom scripts; note their function and dependencies.
Mapping data models: concept-first approach
Before converting data, design the target data model. Don’t try to force old structures into a new system; instead, map concepts.
- Identify core entities: boreholes, logs, layers/strata, samples, lab test results, projects, locations, and attachments.
- For each entity, define required fields and optional fields. Example for a borehole: ID, X/Y (or lat/long), elevation, drilling date, drill method, contractor, site notes.
- Create a mapping table (gINT field → target field). Note unit conversions and controlled vocabularies (e.g., disturbance codes, lithology terms).
- Decide how to handle custom fields or user-defined tables: migrate as-is, map to a generic metadata table, or normalize into structured schema.
- Keep provenance: add migration metadata fields (original gINT ID, migration date, source file path).
Data cleaning and normalization
Migration is an opportunity to improve data quality.
- Normalize units (e.g., convert feet to meters), numeric formats, and measurement precision.
- Standardize coordinate systems: transform coordinates to a common CRS and record the original CRS.
- Consolidate duplicate boreholes: use spatial and attribute matching to identify potential duplicates; merge after human review.
- Harmonize controlled vocabularies: map lithology descriptions to a chosen classification (e.g., ASTM or an internal code list).
- Validate lab test data ranges and flag outliers for review.
- Keep a changelog of automated edits and manual corrections.
Choosing the right target system: alternatives overview
Selection depends on budget, scale, required integrations, and team skills. Below are common classes of alternatives and representative options.
Commercial geotechnical platforms
- gINT Cloud (Bentley): cloud-hosted options and integration within Bentley ecosystem.
- Keynetix: HoleBASE SI — widely used for geotechnical and geoenvironmental database management with migration tools and strong reporting capabilities.
- Leapfrog Works (formerly Seequent Central + Leapfrog): better for 3D geological modelling with data management features.
- Geoscience ANALYTICS platforms (e.g., Rocscience Data Management tools).
Enterprise database + GIS/BIM integration
- Custom solution on SQL Server/PostGIS with a web front-end (Power BI, custom portals, or commercial geotechnical front-ends). Good for organizations that need full control and centralization.
- ArcGIS Online / ArcGIS Enterprise with a geotechnical schema and feature services: strong if GIS integration is essential.
Cloud-native SaaS tools
- Cloud geotechnical data managers that offer multi-user access, REST APIs, and hosted reporting (varies by vendor).
Open-source / hybrid
- PostGIS + QGIS + custom UI or plugins: low-cost, flexible, but requires internal development.
- Geopackage as a portable data container for simplified projects.
Spreadsheet-first workflows
- Excel/Google Sheets + standardized templates + scripts — simplest but less scalable and harder to audit.
Migration methods
There are three broad migration approaches; you may combine them.
-
Automated bulk migration
- Use scripts to parse exported CSV/SQL and insert into the new schema.
- Best for large volumes and standardized data.
- Tools: Python (pandas, SQLAlchemy), FME, ETL platforms.
- Pros: fast and repeatable. Cons: risk of silent mapping errors — require validation.
-
Semi-automated with manual review
- Automate basic field mappings and flag uncertain records for human review.
- Useful when data quality varies and domain expertise is needed for classification.
-
Manual migration / re-entry
- Only for small projects or when historical fidelity is not required.
- Time-consuming but can be used for final normalization.
Suggested process:
- Pilot: migrate a representative subset (5–10 projects) end-to-end, test outputs and templates.
- Iterate: refine mappings, unit conversions, and templates.
- Full run: perform bulk migration in a staged window with backups.
- Validation: automated checks plus spot manual reviews.
- Cutover: freeze writes to the old system, sync late changes, and switch to the new system.
Reporting and template recreation
Reports are often the most visible deliverables; clients expect consistency.
- Recreate templates in the new system early in the pilot so stakeholders can compare outputs.
- If exact visual parity is required, consider generating PDFs from the old system for archival deliverables while using the new system for data and future reports.
- For advanced automation, build templating with tools that support conditional text, loops (for repeating strata), and embedded charts; e.g., Word templates populated via scripts, ReportLab, or vendor report designers.
Integration, APIs, and automation
Modern workflows benefit from APIs and automation.
- Expose data via REST APIs or OGC services (if GIS-enabled).
- Provide a versioned data access layer for downstream consumers (design engineers, GIS teams).
- Automate routine reports, QA checks, and notifications (e.g., when new lab data arrives).
- Consider web-based viewers for logs and boreholes to reduce reliance on desktop software.
Validation, QA/QC, and acceptance
- Define validation rules (coordinate presence, depth continuity, no negative thicknesses, required metadata).
- Run automated validation scripts and generate discrepancy reports.
- Conduct manual acceptance tests with end users comparing legacy and migrated deliverables.
- Keep an issues log and plan for a remediation pass to fix critical problems.
Governance, training, and documentation
- Update data governance policies: who can edit boreholes, lab results, templates, and how backups are handled.
- Provide role-based access control and audit logs if supported.
- Train users with focused sessions: data entry, searching, reporting, and API usage.
- Document the new data model, mapping decisions, and migration steps for future reference.
Common pitfalls and how to avoid them
- Underestimating time to clean and normalize data — budget hands-on review time.
- Ignoring spatial reference systems — always record and transform CRS carefully.
- Losing provenance — preserve original IDs and file references to enable tracing.
- Recreating broken workflows — involve end users early to ensure new tools match their needs.
- Over-customizing the new system immediately — prioritize a stable, maintainable baseline, then extend.
Example migration checklist (compact)
- Inventory files and databases
- Export raw tables (CSV/SQL) and PDFs
- Document custom templates/scripts
- Define target schema and mapping table
- Pilot-migrate representative projects
- Validate and iterate mappings
- Bulk migrate and validate
- Recreate reports and templates
- Train users and cut over
- Archive legacy files and keep read-only access
Cost considerations
- Licensing differences: per-user desktop vs. cloud subscription vs. enterprise DB costs.
- Development and integration: custom ETL, API, and UI development add upfront costs.
- Training and process change: internal ramp-up time and documentation.
- Ongoing maintenance: backups, hosting, and support.
Final recommendations
- Start with a small, representative pilot to validate the migration approach and expected effort.
- Preserve raw data and provenance; treat original gINT exports as the authoritative archive.
- Use a mixed approach: automate what’s consistent, and human-review what’s ambiguous.
- Choose a target that aligns with your broader enterprise architecture (GIS, BIM, cloud).
- Prioritize data quality and governance — the long-term value of migrated data depends on it.
If you’d like, I can:
- produce a sample field mapping spreadsheet from a gINT export you provide, or
- outline a Python ETL script template (pandas + SQLAlchemy) to load CSV exports into a PostGIS schema.
Leave a Reply