How To Backup ArcGIS Enterprise

Posted by on

 

ArcGIS Enterprise Backup Tools


Backing up your ArcGIS Enterprise data is essential for ensuring the integrity and availability of your data. Regular backups safeguard against data loss and provide peace of mind in case of system failure. Below, I’ll walk you through several methods for backing up your ArcGIS Enterprise data.



Method 1. Export Data from Feature Layers

This method involves manually exporting data from feature layers. While it’s an option, it's not typically recommended for large-scale or regular backups due to the manual effort and time required. It’s useful for one-time or ad-hoc backups of specific data.

  • What It Includes: Manual export of individual datasets from specific feature layers, such as those created using Survey123 forms.

  • How It Works: This approach involves exporting feature layers (e.g., shapefiles, CSVs, or GeoJSON files). For Survey123, you may export your survey data from the feature layer it is stored in.

  • Best Use Case:

    • Ideal for small-scale operations or one-time, ad-hoc backups.

    • Helpful for extracting specific datasets for external sharing or analysis.

  • Advantages:

    • Simple to execute with minimal tools.

    • Provides flexibility to target specific datasets, such as a single Survey123 survey result.

  • Limitations:

    • Labor-intensive for frequent or large-scale backups.

    • Lacks automation capabilities.

Method 2: ArcGIS Data Store Backup Tool

The ArcGIS Data Store Backup Tool is a reliable utility for creating secure backups of your ArcGIS Data Store. It helps safeguard against potential data loss and ensures seamless data recovery. 

  • What It Includes: Backups of content stored in ArcGIS Data Store, such as relational data (including Survey123 data), tile caches, and spatiotemporal datasets.

  • How It Works: Using this built-in utility, you can back up specific types of data stores (e.g., relational or spatiotemporal) via command-line operations.

  • Best Use Case:

    • Essential for protecting system-critical data stored in the Data Store, including Survey123 survey responses stored as feature layers.

    • Useful for automated, scheduled backups.

  • Advantages:

    • Supports automation using schedulers like Task Scheduler or cron jobs.

    • Ensures consistent and secure backups of relational survey data.

  • Limitations:

    • Requires familiarity with command-line operations.

    • Limited to data within the Data Store.

        Here's how you can use this tool:

  1. Locate the Backup Tool

    • The backup tool is part of your ArcGIS Data Store installation.

    • Example path on Windows: C:\Program Files\ArcGIS\DataStore\tools

  2. Backup Command References:

Here are some useful commands. Use the following command to execute the backup tool via the command line: 

    • Relational Data Store:
      shell
      exportdatastore --store relational --backupdir <path_to_backup_directory>
      Tile Cache Data Store:
      shell
    • exportdatastore --store tilecache --backupdir <path_to_backup_directory>

    • Spatiotemporal Big Data Store:
      shell
      exportdatastore --store spatiotemporal --backupdir <path_to_backup_directory>

    • Graph Store:
      shell
      exportdatastore --store graph --backupdir <path_to_backup_directory>

    • Object Store:
      shell
      exportdatastore --store object --backupdir <path_to_backup_directory>

    • Restore Data Store:

      shell
      restoredatastore --target relational --backupfile <path_to_backup_file>
    • Components
      1. exportdatastore:
        • This is the command-line utility used to export (create a backup of) an ArcGIS Data Store.
      2. --store object:
        • The --store parameter specifies the type of data store you want to back up. In this case, object refers to the Object Store, which is used to store 3D caches, video BLOBs, and associated metadata files.
      3. --backupdir <path_to_backup_directory>:
        • The --backupdir parameter specifies the directory path where the backup files will be saved. You need to replace <path_to_backup_directory> with the actual path to the directory where you want to store the backup.

  • Schedule the Backup Automate the process using your operating system's task scheduler:

    • On Windows:

      • Open Task Scheduler and create a new task.

      • Add the following command under "Add arguments":

        bash
        /c "C:\Program Files\ArcGIS\DataStore\tools\backupdatastore --store relational --location C:/arcgis/backups"

    • On Linux/Mac:

      • Open the terminal and edit the cron jobs using crontab -e.

      • Example cron job for daily backups at midnight:

        bash
        0 0 * * * /path/to/arcgis/datastore/tools/backupdatastore --store relational --loc

 

Method 3: Database (Enterprise Geodatabase) Backup

 If your data is stored in an enterprise geodatabase like SQL Server or PostgreSQL, using the native backup tools for those databases (such as 

pg_dump 

for PostgreSQL)

 is the recommended method for backing up the geodatabase. These backups will include all data stored within the geodatabase, including Survey123 data.

The backup in this case is not handled by ArcGIS directly but rather by the database system that stores the geospatial data. When you back up the database, you are essentially backing up all of the data stored in the Enterprise Geodatabase, which can include:

    • Feature classes
    • Tables
    • Relationship classes
    • Views
    • Raster datasets
    • Geospatial indexes

  • What It Includes: Complete database backups, covering all geodatabase content, including Survey123 data if stored in an enterprise geodatabase (e.g., SQL Server, PostgreSQL).

  • How It Works: Use native database tools, such as pg_dump for PostgreSQL, to create full or incremental backups of the geodatabase. These backups can be automated for regular intervals.

  • Best Use Case:

    • Ideal for organizations managing Survey123 data in a centralized enterprise geodatabase.

    • Ensures robust data protection by including all database content.

  • Advantages:

    • Centralized and comprehensive backup of spatial and non-spatial data.

    • Database-native tools ensure compatibility and reliability.

  • Limitations:

    • Requires technical knowledge of database management systems.

    • Integration with ArcGIS workflows may need additional configuration.

 

Method 4:  Using WebGIS DR (Disaster Recovery) Tool

ArcGIS Enterprise provides several tools for comprehensive backups of your entire deployment.

  • WebGIS DR (Disaster Recovery) Tool: This tool is used for full deployment backups and is ideal for disaster recovery situations. It allows for scheduling and creating backups of your entire ArcGIS Enterprise system.
  • Backup and Restore Utility: This utility provides flexibility, allowing you to back up specific components or the entire ArcGIS Enterprise deployment. It’s also commonly automated via task schedulers.

  • What It Includes: Backups of the entire ArcGIS Enterprise deployment or specific components, such as:

    • Portal (users, items, groups, etc.)

    • ArcGIS Server configurations and services

    • Data Store (relational and other content, including Survey123 survey data)

    • Web AppBuilder apps and hosted feature layers

  • How It Works: This method involves using tools designed for large-scale, comprehensive backups:

    1. WebGIS Disaster Recovery (DR) Tool:

      • Performs full-system backups covering Portal, ArcGIS Server, and Data Store.

      • Recommended for disaster recovery or system migration scenarios.

    2. Backup and Restore Utility:

      • Provides flexibility to back up individual components or the entire deployment.

      • Can automate the process with scripts and schedulers.

  • Best Use Case:

    • Crucial for organizations requiring complete recovery solutions, especially those reliant on Survey123 services integrated into the enterprise.

    • Suitable for preparing for large-scale disasters or planned migrations.

  • Advantages:

    • Comprehensive and highly customizable.

    • Ensures full recovery of all enterprise components.

  • Limitations:

    • May require expertise in ArcGIS Enterprise system architecture.

    • Setup and maintenance can be complex.

The WebGIS DR Tool is ideal for comprehensive backups of your ArcGIS Enterprise deployment. It ensures that your environment can be restored in case of a disaster. This tool is often used for:

  • Backing up the entire ArcGIS Enterprise (Portal, Server, Data Store, etc.)
  • Performing a complete disaster recovery operation if required

Steps to use the WebGIS DR Tool:

  1. Download and Install WebGIS DR Tool:
    • WebGIS DR Tool is typically part of the ArcGIS Enterprise deployment and can be found in the ArcGIS Enterprise installation directory.
  2. Configure the WebGIS DR Tool:
    • After installation, the tool needs to be configured to back up the components of your ArcGIS Enterprise deployment (Portal, Server, Data Store, etc.).
    • Review the webgisdr.xml configuration file where the settings are defined.
  3. Run the WebGIS DR Tool:
    • You can manually execute the backup by running the tool from the command line.
    • The typical command looks like this:
      css
      webgisdr backup --configFile <path_to_config_file>
    • Ensure that your configuration file (webgisdr.xml) points to the correct environment for backup (e.g., Portal, Server, Data Store, etc.).

Scheduling the Backup with Windows Task Scheduler or Cron Jobs

After you've set up the WebGIS DR Tool for backups, the next step is to schedule these backups to run at regular intervals (e.g., daily, weekly).

For Windows (Using Task Scheduler):

  1. Open Windows Task Scheduler:

    • Open Task Scheduler from the Start menu or by searching for "Task Scheduler."

  2. Create a New Task:

    • Select Create Basic Task.
    • Give it a name like "ArcGIS Enterprise Backup."

  3. Set Trigger for Task:

    • Choose when the task will run (e.g., daily, weekly).
    • Set the time and frequency according to your needs.

  4. Set Action for Task:

    • Choose Start a Program as the action.
    • Browse to the location of your webgisdr.bat or script file.
    • Add any required arguments like the configuration file path, for example:
      swift
      webgisdr.bat backup --configFile "C:\path\to\webgisdr.xml"

  5. Finish and Test the Task:

    • Complete the wizard and test the backup to ensure it’s running as expected.

For Linux (Using Cron Jobs):

  1. Open the Crontab Editor:

    • Run the following command in the terminal to open the crontab editor:
      nginx
      crontab -e

  2. Schedule the Backup:

    • Add a cron job to run the backup script at a regular interval. For example, to run the backup every day at midnight, add the following line:
      css
      0 0 * * * /path/to/webgisdr backup --configFile /path/to/webgisdr.xml

  3. Save and Exit:

    • Save the file and exit the editor (usually by pressing Ctrl + X, then Y to confirm, and Enter to save).

  4. Verify Cron Job:

    • You can check the scheduled jobs with:
      nginx
      crontab -l

 Verify Backups

  • After scheduling, it’s important to verify that the backups are being created correctly.
  • You can check the output logs generated by the WebGIS DR tool for any errors or issues with the backup process.
  • Regularly test the restoration process to ensure that your disaster recovery plan works in case of failure.

Comparison Table of ArcGIS Enterprise Backup Approaches

Approach Description What It Backs Up What It Does Not Include Backup Location Automation Best For Pros Cons
Export Data from Feature Layer Manually export data from ArcGIS Online or Enterprise. Attribute data, spatial data. Attachments, relationships, settings. Local/cloud storage. ❌ No automation. Quick backup of small datasets. ✅ Easy to use.
✅ No database access needed.		 ❌ Time-consuming.
❌ Manual process.
ArcGIS Data Store Backup Tool Backs up ArcGIS Data Store (relational/tile cache). Hosted feature services. Enterprise geodatabase, ArcGIS Server settings. Local/network directory. ✅ Automatable. Federated services in ArcGIS Enterprise. ✅ Ensures data recovery.
✅ Automatable.		 ❌ No ArcGIS Server/Portal backup.
Database Backup (PostgreSQL) Uses `pg_dump` to back up Enterprise Geodatabase. Feature data, relationships, attachments. ArcGIS Server settings, Portal content. .dump file (local/network). ✅ Automatable via Cron Jobs. Enterprise Geodatabases (PostgreSQL). ✅ Captures all spatial data.
✅ Works independently of ArcGIS.		 ❌ External attachments not backed up.
ArcGIS Enterprise Backup Tools Backs up entire ArcGIS Enterprise deployment. ArcGIS Server, Portal, Data Store. Enterprise geodatabase (if external). Local/cloud storage. ✅ Automatable via scripts. Full disaster recovery. ✅ Complete ArcGIS backup.
✅ Automatable.		 ❌ Complex setup.
❌ Large storage required.

Key Insights:

  1. If you only need feature layer data, Approach 1 (Export Data) is the simplest but lacks automation and does not include attachments.
  2. If your data is stored in ArcGIS Data Store, Approach 2 (ArcGIS Data Store Backup Tool) is best, but it does not back up enterprise geodatabases.
  3. If you use PostgreSQL as an Enterprise Geodatabase, Approach 3 (pg_dump) is best for backing up feature classes and attachments.
  4. For a complete ArcGIS Enterprise backup (including Server, Portal, and configurations), Approach 4 (WebGIS DR Tool) is the best solution.

Naba Kumar Chouhan

Comments

Post a Comment