.. -*- coding: utf-8 -*-
..
rest2web header
restindex
page-title: simple is better - JSON-RPC 2.0 Transport: Sockets
crumb: JSON-RPC 2.0 Transport: Sockets
page-description:
JSON-RPC 2.0 Transport: Sockets
/description
tags: simple RPC, JSON, json-rpc, JSON-RPC 2.0, transport, TCP, Unix-Domain-Sockets, netstrings
initialheaderlevel: 2
format: rest
encoding: utf-8
output-encoding: None
file: transport_sockets.txt
/restindex
uservalues
blockquote:
linksource:
site_copyright: webmaster(at)simple-is-better(dot)org
/uservalues
===============================
JSON-RPC 2.0 Transport: Sockets
===============================
:Status: **proposal/draft**
:Date: 2013-05-03
:Author: Roland Koebler
.. contents:: **Table of Contents**
:backlinks: none
.. sectnum::
-----
Using sockets (TCP, UDP or Unix Domain Sockets) with JSON-RPC is simple:
Simply send and receive the JSON-RPC strings. The only problem is to detect
when a Request/Response is complete and where the next Request/Response
starts. Here, three ways are presented:
- shutdown/close the connection after every Request/Response
- encapsulate JSON-RPC in netstrings
- detect JSON-object/array boundaries by using a streaming
JSON-splitter or streaming JSON-decoder
shutdown/close after every Request/Response
-------------------------------------------
Use a new connection for every Request/Response.
The client MUST shutdown the writing (``SHUT_WR``) to signal the end of the Request.
The server MUST close the connection to signal the end of the Response.
This is simple, straight-forward and robust. But it may be slow, since every
Request needs a new connection.
Client-side:
1. open a socket, connect
2. send Request, shutdown socket-writing
3. receive Response (=receive until close/shutdown)
4. close socket
5. goto 1
Server-side:
0. open socket, bind, listen
1. accept a connection
2. receive Request (=receive until shutdown)
3. send Response
4. close connection
5. goto 1
Netstrings
----------
:See:
- http://cr.yp.to/proto/netstrings.txt
- http://en.wikipedia.org/wiki/Netstring
Encode JSON-RPC objects in netstrings.
Since netstrings are a self-delimiting encoding, a single connection can be
used for several Requests/Responses.
But note that the server may close a connection e.g. on error or after a
timeout, so use appropriate error-handling and always use a new connection
as fallback.
Example::
60:{"jsonrpc": "2.0", "method": "first", "params": 42, "id": 1},66:{"jsonrpc": "2.0", "method": "second", "params": [23, 7], "id": 2},
pipelined Requests/Responses / JSON-splitter
--------------------------------------------
It's also possible to use a single connection for several Requests/Responses
without an additional encoding. Then, (a) a streaming JSON-decoder or (b) a
streaming JSON-splitter is needed, which detects the end of a JSON-object/array.
Then, several JSON-RPC Requests can simply be pipelined on a single connection.
Streaming JSON-decoders seem to be rare, but a JSON-splitter which only detects
the end of a JSON-object/array can easily be implemented (see below).
But note that the server may close a connection e.g. on error or after a
timeout, so use appropriate error-handling and always use a new connection
as fallback.
JSON-splitter example in Python 2 (should be easily transferable to other
languages)::
def json_split(s, start=0):
r"""Extract 1st JSON-object/array from a string.
:Returns: (EXTRACTED_JSON_STRING_or_None, i_remainder)
:Raises: ValueError
:Example::
>>> stream = '{"first": "object", "data": "x"} {"second": "object", "data": "y"} ["third", "array"]["fourth", "array"]["incomplete", "arr'
>>> i=0
>>> while True:
... j, i = json_split(stream, i)
... if j is None:
... break
... print j
{"first": "object", "data": "x"}
{"second": "object", "data": "y"}
["third", "array"]
["fourth", "array"]
>>> print stream[i:]
["incomplete", "arr
>>> stream = '{"a": "b", "1": 2, "c": {"1": [1, 2], "3": [{"d": ["}"]}], "2": {"3": 4}}, "xy": "x ] } \\" [ { y"}' * 5
>>> i=0
>>> while True:
... j, i = json_split(stream, i)
... if j is None:
... break
... print j
{"a": "b", "1": 2, "c": {"1": [1, 2], "3": [{"d": ["}"]}], "2": {"3": 4}}, "xy": "x ] } \" [ { y"}
{"a": "b", "1": 2, "c": {"1": [1, 2], "3": [{"d": ["}"]}], "2": {"3": 4}}, "xy": "x ] } \" [ { y"}
{"a": "b", "1": 2, "c": {"1": [1, 2], "3": [{"d": ["}"]}], "2": {"3": 4}}, "xy": "x ] } \" [ { y"}
{"a": "b", "1": 2, "c": {"1": [1, 2], "3": [{"d": ["}"]}], "2": {"3": 4}}, "xy": "x ] } \" [ { y"}
{"a": "b", "1": 2, "c": {"1": [1, 2], "3": [{"d": ["}"]}], "2": {"3": 4}}, "xy": "x ] } \" [ { y"}
"""
state = 0 # 0=search start, 1=array, 11=string inside array
# 2=object, 12=string inside object
i = start # current position in string
b = i # begin of string-part
depth = 0 # array/object-depth-counter
while i < len(s):
# find start of JSON-array or object
if 0 == state:
if s[i] == '[':
state = 1
b = i
depth = 1
elif s[i] == '{':
state = 2
b = i
depth = 1
elif s[i].isspace(): # ignore whitespace
pass
else:
raise ValueError("Invalid character '%s' at %d, expected '[' or '{'." % (s[i], i))
# skip string
elif state > 10:
if s[i] == '\\': # skip char after \
i += 1
elif s[i] == '"': # end of string
state -= 10
# inside array
elif 1 == state:
if s[i] == '"':
state += 10
elif s[i] == '[':
depth += 1
elif s[i] == ']':
depth -= 1
if depth == 0:
return s[b:i+1], i+1
# inside object
elif 2 == state:
if s[i] == '"':
state += 10
elif s[i] == '{':
depth += 1
elif s[i] == '}':
depth -= 1
if depth == 0:
return s[b:i+1], i+1
i += 1
return None, b