Measure:
- Don't block ZServer on uploads and downloads
- Don't hold BLOBS in memory or cache if not necessary (LRU caches tend to break if we split BLOBs in lot of small objects. Size-based caches tend to break on single large objects)
- Transparent for other systems, support normal ZODB operations.
Comments:
- Cache: BLOBs could be cached in a seperate "BLOB" space, e.g. in single files
- Be storage independent?
- Memory efficiency: Storge.load() currently holds all data of an object in a string.
Steps:
simple aspects:
blobs should be known by zodb
storages, esp. clientstorage must be able to recognize blobs
- to avoid putting blob data into the client cache.
blob data mustn't end up in the object cache
blob object and blob data need to be handled separately
blob data on client is stored in temporary files
complicated aspects
- temporary files holding blob data could server as a separated cache for blob data
- storage / zodb api change
Restrictions:
a particular BLOB instance can't be open for read _and_ write at the same time
- Allowed: N readers, no writers; 1 writer, no readers
- Reason:
a writable filehandle opened via a BLOB's 'open' method has a lifetime tied to the transaction in which the 'open' method was called. We do this in order to prevent changes to blob data from "bleeding over" between transactions.