Leightweight Stream Exchange Protocol
Tim Schuster 27605d8aef Merge branch 'develop' of go/lsep into master 2 years ago
examples Removed Profiling Dependency 2 years ago
.gitignore Good number of perf increments 2 years ago
LICENSE Basic Header & Frame Parser 2 years ago
README.md Extended Readme 2 years ago
header.go Good number of perf increments 2 years ago
header_test.go gofmt'd project as final cleanup 2 years ago
socket.go Finished Reader & Added Example Client 2 years ago
tcp_socket.go Added TODO 2 years ago

README.md

LSEP - Lightweight Stream Exchange Protocol

LSEP is a simple and lightweight protocol to exchange streams of bytes with a known length.

This library enables go applications to freely exchange messages over TCP without having to worry about delimiting.

Format

[ Header ] [ Options ] [ Frame Header Length ] [ Length ] [ Data ]

[ Header ] is a 3bit-field that indicate the LSEP version for future upgrades, to ensure that incompatible clients do not interact.

[ Options ] is a 2bit-field to indicate various options for transfer.

[ Frame Header Length ] is a 3bit-field that counts the number of size bytes in the [ Length ] field minus 1.

[ Length ] indicate the total length of the data send.

Overhead Maths

If the Data is 0 bytes long, the entire LSEP packet takes 2 bytes. The first is the Frame Data, the second the 0 byte for the length.

If the Data is under 255 bytes long, the frame takes 2 bytes.

Under 16KiB the header takes 3 bytes.

Under 24MiB the header takes 4 bytes.

Under 4GiB the header takes 5 bytes.

The maximum supported size is 2^64 bytes, utilizing the full 64bit range and the maximum header size of only 9 bytes.

Performance

The LSEP Version as of 22.Aug.16 manages about 1.0 GiB per second over localhost. Most of this is spent allocating the main data buffer, pull requests that improve on this are welcome.

To read data, LSEP allocates the entire required memory at once and then reads into it, avoiding many unnecessary allocations.

Development

LSEP won’t receive any updates unless there is a bug in the implementation that needs fixing, so it will be publishes as is, without any options or versions.

Contributors are welcome to extend and improve the protocol if needed.

LSEP Versions

Version 1 (id:0)

Version 1 supports only Raw Encoding as a Proof-of-Concept, the remaining three options for the header are unused. Other options may be used in the future but are not yet supported.