JFPatch as a Service

API documentation

Introduction

The build service operates on two separate protocols, which drive the underlying technology that provides the RISC OS building environment. The distinction between the protocols allows clients with different needs to get information in a different manners.

The HTTP protocol is useful for clients that want the operation to be performed and get the result, and do not care about what is happening at the server side. The WebSocket protocol is useful for clients that wish to report on the progress, or perform a number of operations in series.

Principles

The system is not complex, but it helps to understand how the service should be used. To function the service must be supplied source code, usually a JFPatch file. JFPatch files were originally intended to be used to patch a separate binary. In such cases, a zip archive can be supplied containing the files that are to be used for the build.

Additional options may (in the future) be given to determine how the build should take place. No such options have been defined in the protocol as yet, so there's nothing to supply to configure the build.

Once the source and any options have been supplied, the build can be started by the client. The cogs turn, archives are unpacked and the build tool (usually JFPatch) is invoked. Any character output is recorded, and sent back to the client as necessary. Throwback is captured and returned to the client as it happens (or buffered for the blocking protocols). Failures (exiting with an error, a non-0 return code, or an abort such as a data abort) cause the build to terminate and will be reported as non-0 return codes to the user. Successful build is recognised by copying the build file to the clipboard, which is then transferred to the client.

Multiple output files are not currently supported.

Examples

Example usage, together with example JFPatch code, can be found in a supporting repository.

HTTP protocol: Blocking HTTP build service

Protocol

The HTTP build service allows building of RISC OS binaries through a POST request, with the response format selectable through the URI.

POST requests are application/x-www-form-urlencoded with the following parameters:

Parameter Meaning
source the source data to build

The response depends on format selected by the URI. The following URIs are supported:

URL Endpoint
/build/binary Method: POST
Outputs a binary file from the build when successful, with the content type application/riscos, supplying the filetype with the name. On a failed build a 400 response will be given with a content type of text/plain, and a body describing the build messages.
/build/json Method: POST
Outputs JSON encoded details of the build and output. The following dictionary keys are defined:
Parameter Meaning
messages A list of build management messages, explaining what the system did to perform the build.
throwback A list of throwback event structures. Each structure is the same format as the 'throwback' server action in the WebSocket protocol.
output A list of text lines output by the build process.
data Base64 encoded binary which was built.
filetype RISC OS file type of the built binary.
rc Return code for the build. Usually 0 for success or 1 for failure.

WebSocket protocol: Streaming protocol

The WebSocket build protocol is based on messages to and from the server. Like the regular HTTP protocol, it must be supplied with the source in order to build. This tool handles the requests in a way that allows a very simple command line invocation of the tool and to obtain its binary output from the clipboard.

Protocol

The server and client communicate through messages. Each message is a JSON encoded list of two items.

When first connected, the server will send a 'welcome' message. Each message from the client will be responded to with either a 'response' or 'error' message. The server may send other messages to the client at any time to explain its progress.

Server actions

Action Meaning
welcome Introduces the server.
Data is the server name and version number.
response Sent as a successful response to an action from the client.
Data is a string giving an indication of how the action was processed.
error Sent as an unsuccessful response to an action from the client.
Data is a message describing the failure.
message Build management message, sent to explain what the system is doing to manage the requested build.
Data is a string explaining the action. All strings are complete statements, without a trailing newline.
output Text produced by the build process itself.
Data is a string from the build. Each string may contain ANSI control characters. The strings will be delivered in a timely manner, but may have been concatenated in order to reduce protocol overheads. The RISC OS environment uses the Latin-1 encoding, which is converted to UTF-8 before transmission from the server - this output will always be in UTF-8.
throwback Information about a throwback event in the build system.
Data is a dictionary containing the following keys:
Key Meaning
reason Throwback reason number (see DDEUtils documentation).
reason_name Human readable string for the reason number (or a number if the reason does not have a name). Usually 'Processing', 'Error' or 'Info'.
filename RISC OS filename to which this event applies.
lineno The line number in the given filename to which thie event applies.
severity Throwback severity number (see DDEUtils documentation).
severity_name Human readable string for the severity number (or a number if the severity does not have a name). Usually 'Error', 'Warning' or 'Serious Error'.
url A 'riscos' scheme URL for the file and line.
message Message reported by this event.
There may be multiple errors or warnings from the build tool. The front end on this site ignores any throwback reasons of 'Processing'.
clipboard Built binary content (this is delivered by a clipboard copy operation internally).
Data is a dictionary containing the following keys:
Key Meaning
filetype RISC OS filetype number for the content.
data Base64 encoded binary data.
rc Return code from the system (usually non-0 if a failure occurred)
Data is a number, usually 0 or 1, but other return codes may be produced by different tools. The environment gives a return code of 125 for a failure with an abort.
complete Declares the build process complete.
Data is a True value.

Client actions

Action Meaning
source Supplies the source code that should be built.
Data is a base64 encoded source to build.
build Requests that the build starts.
Data is ignored.

Example WebSocket communication

The following examples show the interaction between the client and the WebSocket server.

Exchange for a successful build

Sender Action Content
server welcome
'Linking over Internet with RISCOS Pyromaniac Agent version 1.04'
Server announcement and version number.
client source
<base64 source data>
Client supplies source data to be processed.
server response
'Source loaded'
Server acknowledges receipt of the source. This does not mean that the source is buildable by the service.
client build
None
Client requests that the build start.
server response
'Started build'
Server acknowledges the request to start a build and begins processing it.
server message
'Build tool selected: JFPatch'
Server has determined the build tool to use; if the source was not understood a different message would be produced, and the exchange would report the build completion.
server output
'JFPatch ARM assembler v2.56\xdf (02 Mar 2020) [Justin Fletcher]\r\n'
Output from the build tool.
server output
'Pre-processing...\r\n'
server output
'Assembling...\r\n'
server clipboard
{'filetype': 4092, 'data': <base64 data>}
The output file is supplied, together with its RISC OS filetype.
server rc
0
A return code of 0 indicating that the tool exited without error. This is the machine readable format, which is followed by a human-readable message...
server message
'Return code: 0'
server complete
True
The server has finished the build and is now waiting for more source.

Exchange for a successful build

Sender Action Content
server welcome
'Linking over Internet with RISCOS Pyromaniac Agent version 1.04'
Server announcement and version number.
client source
<base64 source data>
Client supplies source data to be processed.
server response
'Source loaded'
Server acknowledges receipt of the source. This does not mean that the source is buildable by the service.
client build
None
Client requests that the build start.
server response
'Started build'
Server acknowledges the request to start a build and begins processing it.
server message
'Build tool selected: JFPatch'
Server has determined the build tool to use; if the source was not understood a different message would be produced, and the exchange would report the build completion.
server output
'JFPatch ARM assembler v2.56\xdf (02 Mar 2020) [Justin Fletcher]\r\n'
Output from the build tool.
server output
'Pre-processing...\r\n'
server output
'Assembling...\r\n'
server throwback
{'severity': 2, 'url': 'riscos:///source#70', 'filename': 'source', 'reason': 1, 'severity_name': 'Serious Error', 'lineno': 70, 'message': 'Unknown or missing variable', 'reason_name': 'Error'}
An error was found whilst trying to assemble the file. The structured data in the throwback report indicates what type of errors were seen.
server output
'Error: Unknown or missing variable (Error number &1a)\n'
Further character output from the build tool, describing the error message.
server rc
1
A return code of 1 indicating that the tool failed. This is the machine readable format, which is followed by a human-readable message...
server message
'Return code: 1'
server complete
True
The server has finished the build and is now waiting for more source.