[fpc-devel] Low-cost encoding reduces PPU size by 10%

[J. Gareth Moreton]

 Hi everyone,

 Sorry if the title sounds like a clickbait you find everywhere on the
Internet, and I apologise again for using FPC as my kind of personal
research ground!
 I've just submitted a patch that uses a cheap, customised encoding system
to minimise the number of bytes required to store fields in PPU files that
are used to store indices, data lengths and offsets... integer values that
are frequently small but nonetheless have a large domain (e.g. LongInt or
QWord).  I've submitted a number of patches here -
https://bugs.freepascal.org/view.php?id=35299 - and the PPU file size is
about 10% smaller.  For large projects like Lazarus, this amounts to over
10 megabytes.
 I admit that storage space isn't a premium on most systems these days, but
for things like embedded systems, then every little helps.

 The encoding system I've named "Varlen" and the technical details can be
found here: http://wiki.freepascal.org/Varlen_Encoding [1]
 The basic idea is that values close to zero take fewer bytes to store than
values near the top of the QWord domain.  Values between 0 and 127, for
example, only require a single byte as opposed to 4 or 8, making it highly
suitable for offsets, counts and length fields, especially if they are
frequently equal to zero.  Conversely, Varlen is not suitable for things
like pointers and hashes, whose literal values can be very large.
 One thing not mentioned but pointed out by another developer is that,
unlike something like LEB128, there is no way for a Varlen to encode an
endless byte stream, so if a PPU file gets corrupted, there's no way it
will put the compiler into an infinite loop or a dangerous resource
starvation situation.

 What do you guys think?
 Gareth aka. Kit
  

Links:
------
[1] http://wiki.freepascal.org/Varlen_Encoding
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.freepascal.org/pipermail/fpc-devel/attachments/20190330/c2684d44/attachment.html>