Reports

Job-Engines

Pegasus Gateways allow for external job-engines to create its reports. A job-engine is essentially a server that receives the request for a report with parameters and user information and builds the report and returns the results - for example files (html, csvs, xlsx, etc.).

The Pegasus Gateway API handles all user authorizations and scheduling, allowing the job-engines to
focus on the reporting logic, in whichever environment they choose.

The job-engine servers that are used can be found in the following endpoint:

[site_url]/job_manager/engines

The response is an array of servers that are used to interact with the report requests that are sent.

{
    "engines": [
        "https://jobs.us1.pegasusgateway.com/jobs-engine",
        "https://jobs.us1.pegasusgateway.com/posted_speed_limits"
    ]
}

Be sure that any of the servers found in this array can be used to generate reports.

You can extend Pegasus with your own job engine to generate your own reports, below are the steps:

Required endpoints

Job engines require 3 endpoints to work

• 1. GET /manifest
• 2. POST /jobs
• 3. GET /jobs/:site_id/:job_id/:file

Manifest

GET [job_engine_url]/manifest

{
	"definitions": {},
	"scripts": []
}

The manifest is a JSON Schema that represents the parameters required for your scripts to work.

The manifest allows UI applications (like Pegasus App) to build forms for creating the reports, as well as scheduling and running them with your gateways /job_manager API

You can use the default engines manifest as a guide

https://d1.pegasusgateway.com/jobs-engine/manifest

definitions

definitions defined on schema

{
	"definitions": {
		"from": {
			"description": "Events starting time",
			"type": "string",
			"format": "date",
			"peg_type": "$from"
		}
	}
}

referencing definitions from the scripts object

"scripts": [
	...
	"properties": {
		"from":
			"$ref: #/definitions/from",
	...
	}
]

You can establish shared definitions that will be used throughout the scripts object.
Definitions represent all the resources that you can work with on the Pegasus API (entities, geofences, triggers, etc), they cannot be modified.

use the JSON schema $ref construct to reference definitions within your script elements.

scripts

{
	"script": "my_report",
	"title": "My Custom Report",
	"desc": "My custom report description",
	"icon": "https://imgur.com/link/to/image.svg",
	"app_url": null,
	"category": "trip_analysis",
	"schedule": true,
	"parameters_schema": {...}
}

The scripts object contains all of the available reports on your engine.

If you define an app_url, when clicking a report an iframe will be embedded with the app_url loaded, otherwise, pegasus loads a default template.

Example with app_url (custom template)

Example without app_url (default template)

category

parameters_schema

script's parameters_schema
[job_engine_url]/manifest

	...
	"parameters_schema": {
		"type": "object",
		"required": ["from", "to"],
		"properties": {
			"from":
				"$ref: #/definitions/from",
			"to":
				"$ref: #/definitions/to",
			"bases":
				"$ref: #/definitions/bases"
			...
		}
	}

The parameters schema tells applications what parameters are required for the script to run.
You define your own schemas, or reference them from your root definitions object.

You can see how pegasus reports section displays the engine's manifest here.

POST to job_engine

Pegasus makes a POST request to your job engine
POST [job_engine_url]/jobs

{
	"configuration": {
		"site_id": "gateway id",
		"site_url": "gateway URL",
		"user_id": "user_id",
		"api_token": "token",
		"tz": "timezone",
		"job_id": 1,
		"job_name": "script_name",
		"job_created_at": timestamp,
		"job_expires": 86400
	},
	"parameters": {
		/* parameters specified by the parameters_schema for this report */
	}
}

{
	"configuration": {
		"site_id": 1,
		"site_url": "https://cloud.pegasusgateway.com",
		"user_id": 86,
		"api_token": "9e34d311fc4ad45d726ab731917554ee279bf23cfa5b148df7c30e01",
		"tz": "UTC",
		"job_id": 22222,
		"job_name": "trips",
		"job_created_at": 1527613165.181
	},
	"parameters": {
		"from": "2018-01-01T00:00:00",
		"to": "2018-01-07T23:59:59",
		"vehicles": [5,6,7],
		"tz": "UTC"
	}
}

This is the endpoint where your magic comes through.
Pegasus Gateway provides your job engine the necessary parameters to construct a report by sending a POST to [job_engine_url]/jobs with the following info:

The configuration object contains a site_url and api_token, which enables you to interact with the Gateway's APIs
(/rawdata, /counters, etc) and generate your report.

The job_expires parameter refers to the duration of the jobs time on the gateway (in seconds), it is relative to the jobs job_created_at timestamp, done_at is marked once the job is updated with result error or done.

Note The api_token is restricted to the user's scopes. You must handle this in your reporting logic (user is asking for a geofence report but does not have access to geofences)

You must use these parameters to report the state of the job back to the gateway

PUT [site_url]/job_manager/:job_id

Updating the Jobs

Update a report's progress:

PUT [site_url]/job_manager/15

{
	"progress": {
		"percentage": 50
	}
}

Update the parameters for a report if a vehicle id (7) was not found for example:

PUT [site_url]/job_manager/15

{
	"parameters": {
		"from": "2018-01-01T00:00:00",
		"to": "2018-01-07T23:59:59",
		"vehicles": [5, 6],
		"tz": "UTC"
}

Update the state to done

PUT [site_url]/job_manager/15

{
	"state": "done"
}

Update the job when report is done

PUT [site_url]/job_manager/15

{
	"parameters": {},
	"progress": {
		"percentage": 100
	},
	"state": "done",
	"result": {
		"files": ["custom.json", "custom.csv", "custom.xlsx", "custom.pdf"],
		"email": {
			"files": ["custom.xlsx", "custom.pdf"],
			"message": "<h2>Custom Report</h2><br><p>Attached detail with scheduled custom report</p>",
			"subject": "Custom Report"
		},
		"error": ""
	}
}

Report returns an error message

PUT [site_url]/job_manager/15

{
	"parameters": { ... },
	"progress": {
		"percentage": 60
	},
	"state": "error",
	"result": {
		"error": "Failed to execute rawdata request"
	}
}

You need to inform the Gateway that started your Job of its progress and available data,
to do so you can make PUT requests with the api_token given in the configuration block.

Note site_url refers to the pegasus site url

PUT [site_url]/job_manager/:job_id

The payload can contain one or all of the following:

• parameters: updated parameters (i.e post-validation)
• progress: progress for the job.
• state: state of the job (running, error or done)
• result:
  • files: array with files generated in the report
  • email:
    • files: array with files to send in the email
    • subject: email subject
    • message: email subject message
  • error: error message when the script fails (optional)
  • summary:

When the report is done, you must update the job with other parameters, in this case with the result field, this field has the files before explained.

Updating the job state to error or done closes the job to any further updates.
You won't be able to modify it further.

Getting data from Jobs

GET [site_url]/job_manager/15/script_name.json
results in
[job_egine_url]/jobs/1/15/script.json

{
	"vid": 5,
	"name": "Test 1",
	"from": "2018-01-01T00:00:00",
	"to": "2018-01-07T23:59:59",
	"idleTime": 258929,
	"primary_group": null,
	"resolved": true,
	"percent_idle": 57
},
{
	"vid": 6,
	"name": "Test 2",
	"from": "2018-01-01T00:00:00",
	"to": "2018-07-07T23:59:59",
	"idleTime": 57911,
	"primary_group": null,
	"resolved": true,
	"percent_idle": 25
}

Once your report is finished, This is endpoint used by pegasus to fetch the files generated:

GET [site_url]/jobs/:site_id/:job_id

For example,

GET [site_url]/job_manager/15/script_name.json

results in:

[job_engine_url]/jobs/1/15/script.json

Pegasus Job Manager

[site_url]/job_manager/engines

[
	"https://d1.pegasusgateway.com/job-engines",
	"..."
]

Each gateway has a /job_manager API designed to interact with the job-engines registered on the gateway

You can view what job-engines are available on [site_url]/job_manager/engines

Making the POST request to generate the report

Create a report
POST [site_url]/job_manager

{
    "script": "name",
    "name" : "report name",
    "expires": 0,
    "engine": "engine_name",
    "args" : { ... }
}

example creating an idling report from the d1 manifest

POST [site_url]/job_manager

{
  "script": "idling",
  "name": "Idling Usage",
  "expires": 432000,
  "engine": "{{JOB-ENGINE-URL}}", // be sure to use the appropriate job engine-url
  "args": {
    "from": "2018-07-10T00:00:00",
    "to": "2018-07-10T23:59:59",
    "vehicles": [],
    "groups": [
      500
    ],
    "prefs": {
      "distance": "mile",
      "speed": "mph",
      "language": "es",
      "volume": "gallon"
    },
    "tz": "UTC"
  }
}

RESPONSE

{
    "engine": "{{JOB-ENGINE-URL}}",
    "transaction": null,
    "name": "Idling Usage",
    "script": "idling",
    "created_at": 1532031389.04,
    "expires": 432000,
    "state": "created",
    "__v": 2,
    "params": {
        "tz": "UTC",
        "vehicles": [
            1673
        ],
        "to": "2018-07-10T23:59:59",
        "groups": [],
        "from": "2018-07-10T00:00:00",
        "prefs": {
            "volume": "gallon",
            "distance": "mile",
            "speed": "mph",
            "language": "es"
        }
    },
    "result": {},
    "done_at": null,
    "owner": 54,
    "progress": {},
    "cache_key": null,
    "id": 251336
}

To create a report pegasus sends a payload to the /job_manager resource [site_url]/job_manager with the following info:

Scheduling

Scheduling a report
POST [site_url]/tasks

{
  "id": "task_id",
  "name": "task_name",
  "processes": [
    [
      {
        "name": "report_name",
        "script": "script",
        "tz": "timezone",
        "engine": "job_engine_url"
      },
      {
        "vehicles": [
          5,
          6
        ],
        "recipients": [
          "[email protected]"
        ],
        "user": [
          11
        ],
        "groups": [],
        "back": {
          "days": 7,
          "hours": 0
        },
        "tz": "UTC"
      }
    ]
  ],
  "schedule": []
}

Custom report task
POST [site_url]/tasks

{
	"name": "Custom Report Task",
	"processes": [
		{
			"name":"Custom Report",
			"script":"my_script",
			"tz": "UTC",
			"engine":"http://mycustomjobengine.com"
		},
		{
			"vehicles": [5,6],
			"recipients": ["[email protected]"],
			"users": [11],
			"groups":[],
			"back": {
				"days": 7,
				"hours": 0
			},
			"tz": "UTC"
		}
	],
	"schedule": ["0-15 9 * * 1"]
}

Jobs can additionally be scheduled, with the Pegasus /tasks API

Task parameters:

Configuration parameters:

Parameters: