RISC OS Build service: API docs

Introduction

The build service operates on two separate protocols, which drive the underlying technology that provides the RISC OS building environment. The distinction between the protocols allows clients with different needs to get information in a different manners.

HTTP protocol: The HTTP protocol provides a blocking interface, where the client sends a POST request to the server, and there is no response until the build has completed. The protocol also has two variants - one where it responds with only the built binary (or an error report), and a second where it returns a structured JSON response from the build.
WebSocket protocol: The Web Socket protocol provides a streaming interface, where the client connects to the service, then issues commands to direct it to perform actions, and the server responds to those actions in real time.

The HTTP protocol is useful for clients that want the operation to be performed and get the result, and do not care about what is happening at the server side. The WebSocket protocol is useful for clients that wish to report on the progress, or perform a number of operations in series.

Principles

The system is not complex, but it helps to understand how the service should be used. To function the service must be supplied source code, usually a JFPatch file. JFPatch files were originally intended to be used to patch a separate binary. In such cases, a zip archive can be supplied containing the files that are to be used for the build.

Additional options may (in the future) be given to determine how the build should take place. No such options have been defined in the protocol as yet, so there's nothing to supply to configure the build.

Once the source and any options have been supplied, the build can be started by the client. The cogs turn, archives are unpacked and the build tool (usually JFPatch) is invoked. Any character output is recorded, and sent back to the client as necessary. Throwback is captured and returned to the client as it happens (or buffered for the blocking protocols). Failures (exiting with an error, a non-0 return code, or an abort such as a data abort) cause the build to terminate and will be reported as non-0 return codes to the user. Successful build is recognised by copying the build file to the clipboard, which is then transferred to the client.

Multiple output files are not currently supported.

Examples

Example usage, together with example JFPatch code, can be found in a supporting repository.

HTTP protocol: Blocking HTTP build service

Protocol

The HTTP build service allows building of RISC OS binaries through a POST request, with the response format selectable through the URI.

The HTTP protocol can be found at http://build.riscos.online/ and http://json.build.riscos.online/. The former has a timeout of 1 minute, whilst the latter should be unlimited. The services are also available on https.

POST requests are application/x-www-form-urlencoded with the following parameters:

Parameter	Meaning
`source`	the source data to build

The response depends on format selected by the URI. The following URIs are supported:

URL Endpoint

/build/binary Method: POST
Outputs a binary file from the build when successful, with the content type application/riscos, supplying the filetype with the name. On a failed build a 400 response will be given with a content type of text/plain, and a body describing the build messages.

/build/json

Method: POST
Outputs JSON encoded details of the build and output. The following dictionary keys are defined:

Parameter	Meaning
`messages`	A list of build management messages, explaining what the system did to perform the build.
`throwback`	A list of throwback event structures. Each structure is the same format as the 'throwback' server action in the WebSocket protocol.
`output`	A list of text lines output by the build process.
`data`	Base64 encoded binary which was built.
`filetype`	RISC OS file type of the built binary.
`rc`	Return code for the build. Usually 0 for success or 1 for failure.

WebSocket protocol: Streaming protocol

The WebSocket build protocol is based on messages to and from the server. Like the regular HTTP protocol, it must be supplied with the source in order to build. This tool handles the requests in a way that allows a very simple command line invocation of the tool and to obtain its binary output from the clipboard.

Protocol

The WebSocket protocol can be found at http://build.riscos.online/build/ws. The service is also available on https.

The server and client communicate through messages. Each message is a JSON encoded list of two items.

The first element (the 'action') of the list indicates how the message should be processed.
The second element (the 'data') contains any clarifying content for the action. Usually this is a string, but it may be a data structure for some actions.

When first connected, the server will send a 'welcome' message. Each message from the client will be responded to with either a 'response' or 'error' message. The server may send other messages to the client at any time to explain its progress.

Server actions

Action Meaning

welcome Introduces the server.
Data is the server name and version number.

response Sent as a successful response to an action from the client.
Data is a string giving an indication of how the action was processed, or a list of a string indicating the action followed by associated data.

error Sent as an unsuccessful response to an action from the client.
Data is a message describing the failure.

message Build management message, sent to explain what the system is doing to manage the requested build.
Data is a string explaining the action. All strings are complete statements, without a trailing newline.

output Text produced by the build process itself.
Data is a string from the build. Each string may contain ANSI control characters. The strings will be delivered in a timely manner, but may have been concatenated in order to reduce protocol overheads. The RISC OS environment uses the Latin-1 encoding, which is converted to UTF-8 before transmission from the server - this output will always be in UTF-8.

throwback

Information about a throwback event in the build system.
Data is a dictionary containing the following keys:

Key	Meaning
`reason`	Throwback reason number (see DDEUtils documentation).
`reason_name`	Human readable string for the reason number (or a number if the reason does not have a name). Usually 'Processing', 'Error' or 'Info'.
`filename`	RISC OS filename to which this event applies.
`lineno`	The line number in the given filename to which thie event applies.
`severity`	Throwback severity number (see DDEUtils documentation).
`severity_name`	Human readable string for the severity number (or a number if the severity does not have a name). Usually 'Error', 'Warning' or 'Serious Error'.
`url`	A 'riscos' scheme URL for the file and line.
`message`	Message reported by this event.

There may be multiple errors or warnings from the build tool. The front end on this site ignores any throwback reasons of 'Processing'.

clipboard

Built binary content (this is delivered by a clipboard copy operation internally).
Data is a dictionary containing the following keys:

Key	Meaning
`filetype`	RISC OS filetype number for the content.
`data`	Base64 encoded binary data.

rc Return code from the system (usually non-0 if a failure occurred)
Data is a number, usually 0 or 1, but other return codes may be produced by different tools. The environment gives a return code of 125 for a failure with an abort.

complete Declares the build process complete.
Data is a True value.

Client actions

Action Meaning

source Supplies the source code that should be built.
Data is a base64 encoded source to build.

build Requests that the build starts.
Data is ignored.

options Requests information about the server options.
Data is ignored. The response will include a map of the options supported by the server and their current values (for this session). See the 'option' action, below, for details of the options which are currently supported.

option

Change the value of an option.
Data is a list containing two items - the option name, and the new value. If the option cannot be changed or its value is invalid an error will be returned. The currently supported options are:

Option	Meaning
`timeout`	Specifies the timeout for the build process in seconds.
`ansitext`	Controls whether the ANSI text translation is performed (converts the RISC OS VDU sequences into ANSI terminal control codes). True enables the ANSI text mode (the default), whilst False disables it.

Example WebSocket communication

The following examples show the interaction between the client and the WebSocket server.

WebSocket exchange for a successful build

Sender	Action	Content
server	`welcome`	'Linking over Internet with RISCOS Pyromaniac Agent version 1.04' Server announcement and version number.
client	`source`	<base64 source data> Client supplies source data to be processed.
server	`response`	'Source loaded' Server acknowledges receipt of the source. This does not mean that the source is buildable by the service.
client	`build`	None Client requests that the build start.
server	`response`	'Started build' Server acknowledges the request to start a build and begins processing it.
server	`message`	'Build tool selected: JFPatch' Server has determined the build tool to use; if the source was not understood a different message would be produced, and the exchange would report the build completion.
server	`output`	'JFPatch ARM assembler v2.56\xdf (02 Mar 2020) [Justin Fletcher]\r\n' Output from the build tool.
server	`output`	'Pre-processing...\r\n'
server	`output`	'Assembling...\r\n'
server	`clipboard`	{'filetype': 4092, 'data': <base64 data>} The output file is supplied, together with its RISC OS filetype.
server	`rc`	0 A return code of 0 indicating that the tool exited without error. This is the machine readable format, which is followed by a human-readable message...
server	`message`	'Return code: 0'
server	`complete`	True The server has finished the build and is now waiting for more source.

WebSocket exchange for a build with errors

Sender	Action	Content
server	`welcome`	'RISC OS Build System version 2.0.109' Server announcement and version number. The original version reported itself as `'Linking over Internet with RISCOS Pyromaniac Agent version 1.04'`, a reference to its release on April 1st. The current versions report the version of the build system which is used.
client	`source`	<base64 source data> Client supplies source data to be processed.
server	`response`	'Source loaded' Server acknowledges receipt of the source. This does not mean that the source is buildable by the service.
client	`build`	None Client requests that the build start.
server	`response`	'Started build' Server acknowledges the request to start a build and begins processing it.
server	`message`	'Build tool selected: JFPatch' Server has determined the build tool to use; if the source was not understood a different message would be produced, and the exchange would report the build completion.
server	`output`	'JFPatch ARM assembler v2.56\xdf (02 Mar 2020) [Justin Fletcher]\r\n' Output from the build tool.
server	`output`	'Pre-processing...\r\n'
server	`output`	'Assembling...\r\n'
server	`throwback`	{'severity': 2, 'url': 'riscos:///source#70', 'filename': 'source', 'reason': 1, 'severity_name': 'Serious Error', 'lineno': 70, 'message': 'Unknown or missing variable', 'reason_name': 'Error'} An error was found whilst trying to assemble the file. The structured data in the throwback report indicates what type of errors were seen.
server	`output`	'Error: Unknown or missing variable (Error number &1a)\n' Further character output from the build tool, describing the error message.
server	`rc`	1 A return code of 1 indicating that the tool failed. This is the machine readable format, which is followed by a human-readable message...
server	`message`	'Return code: 1'
server	`complete`	True The server has finished the build and is now waiting for more source.