Create an agent to run Qt Core benchmarks

A runner needs to be developed to execute benchmarks contained in benchmarks\tests - qt/qtbase.git - Qt Base (Core, Gui, Widgets, Network, ...) on physical hardware in a controlled environment.

The agent needs to connect via socket.io API to https://qt-cherry-pick-bot.herokuapp.com

In the suggested communications schema, the agent connects to the host and requests work:

Table shows SEND type events. (If the Agent sends an event 'connect', the Host should receive this event)

Event decriptions:

Connect

The Agent sends a 'connect' request to the Host including an auth header. The auth header should include the hostname and a secret token.

{auth: {clientType: "agent", hostname: "myhostname1", secret: "someauthtoken"}}

statusUpdate

Following a successful connection, the Agent should automatically send a statusUpdate event with a Work object describing the current job, if any. An empty object should be sent if no work is running.

statusUpdate should also be sent unsolicited when:

1. When a new job is received and moves into the 'new' status.
2. The currently running job changes status state.
3. When the runner prepares to run the next test in the suite. 

JS example class for a work item:
class Work {
  constructor(o={}) {
    this.integrationId = o.integrationId;
    this.integrationURL = o.integrationURL,
    this.integrationTimestamp = o.integrationTimestamp;
    this.sha = o.sha;
    this.status = o.status || Status.Idle;
    this.detailMessage = o.detailMessage;
    this.updateTimestamp = o.updateTimestamp;
    this.integrationData = o.integrationData;
  }
}

Allowed statuses:
new, git, configure, build, test, results, done, idle

fetchWork

When the Agent starts up, finishes a job, or on an interval of 30 seconds while idling, the 'fetchWork' signal should be sent to the host to request new work. No request body.

queryWork (DEV-Mode only):

queryWork can be emitted by an agent to non-destructively request the next work item in queue. This function should not be available in production.

mockItem (DEV-Mode only):

An agent emitting the 'mockItem' cause the server to generate a valid mock work item if the agent's queue is empty. An agent can this repeatedly the oldest queued item with 'queryWork', or fetch it and clear it from queue with 'fetchWork' as normal.

sendWork

The Host sends a 'sendWork' signal only in response to a 'fetchWork' signal.

{
    integrationId: '189470895',
    sha: '69609f0b3575c571fff4564f6f16cbca9c7d81f6'
}

requestStatus (Probably not needed)

The Host may request the status of an Agent at any time. The agent should respond with the 'statusUpdate' signal and message body as above. The Host will avoid performing 'requestStatus' while the Agent reports that it is in status 'test' for a given timeout to allow reasonable time for tests to complete.

Suggested internal schema for test results. The Agent should write these results directly to the database at testresults.qt.io/influxdb after all tests have completed.

{
    integrationId: "189470895",
    integrationTimestamp, "Fri Feb 10 2023 13:11:13",
    hostname: "hostname1",
    platform: "Ubuntu_20.04",
    executedTests:
    [
        {
        test_name: "tst_QString",
        test_function: "concatenation",
        result: 16.352,
        }
    ],
    crashed: [
    {
        test_name: "tst_OtherTest",
        log: ""  // Should somehow be made available for review.
    }
    ]
}

Suggested schema for writing point results to Influxdb:

{
    measurement: 'benchmark',
    tags: {
        platform: 'Ubuntu 20.04',
        host: 'myhostname1',
        branch: 'dev',
        gcc: '10.0.3',
        cpu_model: 'Intel Core i7-7700K',
        case_name: 'concatenation',
        test_name: 'tst_QString',
        data_row: '12876363564',  // Unique data input for this execution provided by the test case.
    },
    fields: {
        crashed: false,
        value: 188423,  // Test result. (in ms?)
        iterations: 200,  // How many times the test case ran
        integration_id: '189470895',
        sha1: "69609f0b3575c571fff4564f6f16cbca9c7d81f6"
    },
    time: '1434055562000000000' // Unix nanosecond timestamp. Use the timestamp of the integration, not when the individual test was executed.
}