Skip to main content
Skip table of contents

Data Pipeline Monitoring, File Downloading and Logging

To use this feature, you must have the System Administrator access role.

Please reach out to your customer success and support teams for further assistance.

The File Delivery Status page provides access to original and transformed files, as well as information and error logs to help troubleshoot file delivery, processing and loading issues.

File Status

  1. Navigate to Admin Data Flow Automation

  2. The system will navigate to the landing page of Data Flow Automation, which is the Data Services page.

  3. From Data Services page, navigate to File Delivery Status.

  1. The File Delivery Status grid shows the following columns:

    1. Filename - the original filename of the file

    2. Delivered On - the date and time the file was shared by the client on Amazon S3

    3. Delivered From - the name of the data source used in the data pipeline

    4. Delivered Via - the name of the data service type used in the data pipeline’s data source, e.g. Amazon S3

    5. Delivery Status -

      1. (tick) if the file has been downloaded successfully from the data source

      2. (warning) if downloading the file from the data source encountered an error

    6. Processing Status

      1. (tick) if the file requires transformation and was successfully transformed, and ready for file loading

      2. (tick) if the file does not require transformation but was successfully pushed for file loading

      3. ⏳ if the file has been transmitted to IBM App Connect but the transformed copy is not yet available

      4. (warning) if transmission or transformation of file encountered error

    7. Loading Status

      1. (tick) if the file has loaded successfully

      2. (warning) if the file encountered an error while loading

  2. On the File Delivery Status page, click on the preview panel icon to see details of the pipeline which processed each file. This is useful to verify the filename pattern which the file matched on.

File Download

Downloading a copy of the original file

  1. On the File Delivery Status page, choose a file

  2. Right-click and select Download Original File from the action menu

  3. The copy of the file will start downloading on browser

Downloading a copy of the transformed file

  1. On the File Delivery Status page, choose a file that has been transformed

  2. Right-click and select Download Transformed Copy from the action menu

  3. The copy of the transformed file will start downloading on the browser

Data Pipeline Logs

The File Delivery Logs page shows list of events that happened when processing the file.

  1. From the File Delivery Status page, select a file and right-click and choose Logs from the action menu.

  2. The File Delivery Logs page shows the following information:

    1. Log Type

    2. Event On - date and time the process happened

    3. Event - a description of the event

    4. Pipeline Name - pipeline that were used to process the file

    5. Service Name

    6. Service Type

    7. Filename Pattern

    8. Target System

    9. Filename

  1. Additional log details, such as source and destination of files can be seen on the File Details preview panel on the right

JavaScript errors detected

Please note, these errors can depend on your browser setup.

If this problem persists, please contact our support.