Package org.apache.uima.cas.impl
Class BinaryCasSerDes6
java.lang.Object
org.apache.uima.cas.impl.BinaryCasSerDes6
User callable serialization and deserialization of the CAS in a compressed Binary Format
This serializes/deserializes the state of the CAS. It has the capability to map type systems,
so the sending and receiving type systems do not have to be the same.
- types and features are matched by name, and features must have the same range (slot kind)
- types and/or features in one type system not in the other are skipped over
Header specifies to reader the format, and the compression level.
How to Serialize:
1) create an instance of this class
a) if doing a delta serialization, pass in the mark and a ReuseInfo object that was created
after deserializing this CAS initially.
b) if serializaing to a target with a different type system, pass the target's type system impl object
so the serialization can filter the types for the target.
2) call serialize() to serialize the CAS
3) If doing serialization to a target from which you expect to receive back a delta CAS,
create a ReuseInfo object from this object and reuse it for deserializing the delta CAS.
TypeSystemImpl objects are lazily augmented by customized TypeInfo instances for each type encountered in
serializing or deserializing. These are preserved for future calls, so their setup / initialization is only
needed the first time.
TypeSystemImpl objects are also lazily augmented by typeMappers for individual different target typesystems;
these too are preserved and reused on future calls.
Compressed Binary CASes are designed to be "self-describing" -
The format of the compressed binary CAS, including version info,
is inserted at the beginning so that a proper deserialization method can be automatically chosen.
Compressed Binary format implemented by this class supports type system mapping.
Types in the source which are not in the target
(or vice versa) are omitted.
Types with "extra" features have their extra features omitted
(or on deserialization, they are set to their default value - null, or 0, etc.).
Feature slots which hold references to types not in the target type system are replaced with 0 (null).
How to Deserialize:
1) get an appropriate CAS to deserialize into. For delta CAS, it does not have to be empty, but it must
be the originating CAS from which the delta was produced.
2) If the case is one where the target type system == the CAS's, and the serialized for is not Delta,
then, call aCAS.reinit(source). Otherwise, create an instance of this class -%gt; xxx
a) Assuming the object being deserialized has a different type system,
set the "target" type system to the TypeSystemImpl instance of the
object being deserialized.
a) if delta deserializing, pass in the ReuseInfo object created when the CAS was serialized
3) call xxx.deserialize(inputStream)
Compression/Decompression
Works in two stages:
application of Zip/Unzip to particular sub-collections of CAS data,
grouped according to similar data distribution
collection of like kinds of data (to make the zipping more effective)
There can be up to ~20 of these collections, such as
control info, float-exponents, string chars
Deserialization:
Read all bytes,
create separate ByteArrayInputStreams for each segment
create appropriate unzip data input streams for these
Slow but expensive data:
extra type system info - lazily created and added to shared TypeSystemImpl object
set up per type actually referenced
mapper for type system - lazily created and added to shared TypeSystemImpl object
in identity-map cache (size limit = 10 per source type system?) - key is target typesystemimpl.
Defaulting:
flags: doMeasurements, compressLevel, CompressStrategy
Per serialize call: cas, output, [target ts], [mark for delta]
Per deserialize call: cas, input, [target ts], whether-to-save-info-for-delta-serialization
CASImpl has instance method with defaulting args for serialization.
CASImpl has reinit which works with compressed binary serialization objects
if no type mapping
If type mapping, (new BinaryCasSerDes6(cas,
marker-or-null,
targetTypeSystem (for stream being deserialized),
reuseInfo-or-null)
.deserialize(in-stream)
Use Cases, filtering and delta
**************************************************************************
* (de)serialize * filter? * delta? * Use case
**************************************************************************
* serialize * N * N * Saving a Cas,
* * * * sending Cas to service with identical ts
**************************************************************************
* serialize * Y * N * sending Cas to service with
* * * * different ts (a guaranteed subset)
**************************************************************************
* serialize * N * Y * returning Cas to client
* * * * uses info saved when deserializing
* * * * (?? saving just a delta to disk??)
**************************************************************************
* serialize * Y * Y * NOT SUPPORTED (not needed)
**************************************************************************
* deserialize * N * N * reading/(receiving) CAS, identical TS
**************************************************************************
* deserialize * Y * N * reading/receiving CAS, different TS
* * * * ts not guaranteed to be superset
* * * * for "reading" case.
**************************************************************************
* deserialize * N * Y * receiving CAS, identical TS
* * * * uses info saved when serializing
**************************************************************************
* deserialize * Y * Y * receiving CAS, different TS (tgt a feature subset)
* * * * uses info saved when serializing
**************************************************************************
-
Nested Class Summary
Nested ClassesModifier and TypeClassDescriptionprivate class
static enum
Compression alternativesstatic enum
private class
Modified Values Modified heap values need fsStartIndexes conversionstatic class
Info reused for 1) multiple serializations of same cas to multiple targets (a speedup), or 2) for delta cas serialization, where it represents the fsStartIndex info before any mods were done which could change that info, or 3) for deserializing with a delta cas, where it represents the fsStartIndex info at the time the CAS was serialized out..private class
Modified Values Output: For each FS that has 1 or more modified values, write the heap addr converted to a seq # of the FS For all modified values within the FS: if it is an aux array element, write the index in the aux array and the new value otherwise, write the slot offset and the new value -
Field Summary
FieldsModifier and TypeFieldDescriptionprivate AllowPreexistingFS
Things for just deserializationprivate DataInputStream
private static final int
private final ByteArrayOutputStream[]
private DataInputStream
private DataOutputStream
private static final int
private final ByteHeap
private final CASImpl
Things for both serialization and Deserializationprivate final BinaryCasSerDes6.CompressLevel
private final BinaryCasSerDes6.CompressStrat
private DataInputStream
private DataOutputStream
private static final int
private final DataInputStream[]
private static final long
private final boolean
private DataInputStream
private final boolean
private final DataOutputStream[]
private DataInputStream
private DataOutputStream
private static final int
private DataInputStream
private DataOutputStream
private static final int
private int
private final int[]
private IntVector
private DataInputStream
private DataOutputStream
private static final int
private DataInputStream
private DataOutputStream
private static final int
private BitSet
private int[]
private BitSet
private DataInputStream
private DataOutputStream
private static final int
private final CasSeqAddrMaps
private int[]
private int
private DataInputStream
private static final int
private int
private final Inflater[]
private DataInputStream
private static final int
private static final int[]
private int
private final int[]
private boolean
private boolean
private final boolean
private boolean
private boolean
Things that are used by common routines among serialization and deserializationprivate boolean
private DataInputStream
private static final int
private DataInputStream
private static final int
private final LongHeap
private int
These indexes remember sharable common values in aux heaps Values must be in aux heap, but not part of arrays there so that rules out boolean, byte, and shortsprivate final MarkerImpl
private boolean
private OptimizeStrings
private final int[][]
Hold prev instance of FS which have non-array FSRef slots, to allow computing these to match case where a 0 value is used because of type filtering for each typecode, only set if the type has 1 or more non-array fsref set only for non-filtered domain types set only for non-0 values if fsRef is to filtered type, value serialized will be 0, but this slot not set On deserialization: if value is 0, skip settingprivate String[]
private final boolean
private int[]
private DataOutputStream
Things for just serializationprivate DataInputStream
private static final int
private final ShortHeap
private final SerializationMeasures
private DataInputStream
private static final int
private final StringHeap
private int
private DataInputStream
private DataOutputStream
private static final int
private DataInputStream
private DataOutputStream
private static final int
private DataInputStream
private DataOutputStream
private static final int
private final TypeSystemImpl
private final IntVector
private int
private static final boolean
private static final boolean
private static final boolean
private TypeSystemImpl
Things set up for one instance of this classprivate DataInputStream
private DataOutputStream
private static final int
private TypeSystemImpl.TypeInfo
private final CasTypeSystemMapper
private CasTypeSystemMapper
private int
private static final int
Version of the serializer/deserializer, used to allow deserialization of older versions Version 0 - initial SVN checkin Version 1 - changes to support CasTypeSystemMapper -
Constructor Summary
ConstructorsConstructorDescriptionSetup to serialize (not delta) or deserialize (not delta) using binary compression, no type mapping but only processing reachable Feature StructuresSetup to serialize (not delta) or deserialize (maybe delta) using binary compression, no type mapping and only processing reachable Feature StructuresBinaryCasSerDes6
(AbstractCas cas, MarkerImpl mark, TypeSystemImpl tgtTs, BinaryCasSerDes6.ReuseInfo rfs) Setup to serialize (maybe delta) or deserialize (maybe delta) using binary compression, with type mapping and only processing reachable Feature StructuresBinaryCasSerDes6
(AbstractCas cas, MarkerImpl mark, TypeSystemImpl tgtTs, BinaryCasSerDes6.ReuseInfo rfs, boolean doMeasurements) Setup to serialize (maybe delta) or deserialize (maybe delta) using binary compression, with type mapping and only processing reachable Feature Structures, output measurementsBinaryCasSerDes6
(AbstractCas aCas, MarkerImpl mark, TypeSystemImpl tgtTs, BinaryCasSerDes6.ReuseInfo rfs, boolean doMeasurements, BinaryCasSerDes6.CompressLevel compressLevel, BinaryCasSerDes6.CompressStrat compressStrategy) Setup to serialize or deserialize using binary compression, with (optional) type mapping and only processing reachable Feature StructuresBinaryCasSerDes6
(AbstractCas cas, TypeSystemImpl tgtTs) Setup to serialize (not delta) or deserialize (not delta) using binary compression, with type mapping and only processing reachable Feature Structures -
Method Summary
Modifier and TypeMethodDescriptionprivate void
addStringFromFS
(TypeSystemImpl.TypeInfo srcTypeInfo, int iSrcHeap, int tCode) private void
private void
Method: write with deflation into a single byte array stream skip if not worth deflating skip the Slot_Control stream record in the Slot_Control stream, for each deflated stream: the Slot index the number of compressed bytes the number of uncompressed bytes add to header: nbr of compressed entries the Slot_Control stream size the Slot_Control stream all the zipped streamsboolean
compareCASes
(CASImpl c1, CASImpl c2) Compare 2 CASes, with perhaps different type systems.private long
decodeDouble
(long mants, int exponent) private int
decodeIntSign
(int v) void
deserialize
(InputStream istream) void
deserialize
(InputStream istream, AllowPreexistingFS allowPreexistingFS) Version used by uima-as to read delta cas from remote parallel stepsvoid
deserializeAfterVersion
(DataInputStream istream, boolean isDelta, AllowPreexistingFS allowPreexistingFS) private int
encodeIntSign
(int v) private void
enqueueFeatures
(int addr) Enqueue all FSs reachable from features of the given FS.private void
enqueueFS
(int fsAddr) private DataInput
private int
getPrevIntValue
(int iHeap, int offset) private static int
incrToNextFs
(int[] heap, int iHeap, TypeSystemImpl.TypeInfo typeInfo) private int
Serializing: Called at beginning of serialize, scans whole CAS or just delta CAS If doing delta serialization, fsStartIndexes is passed in, pre-initialized with a copy of the map info below the line.private int[]
initPrevIntValue
(int iHeap) private boolean
isInstanceInTgtTs
(int fsAddr) private static DataOutputStream
(package private) String
printCasInfo
(CASImpl cas) private int
processFsxPart
(int[] fsIndexes, int fsNdxStart, boolean isDoingEnqueue, boolean isWrite) private void
processIndexedFeatureStructures
(CASImpl cas, boolean isWrite) private void
private int
private void
readByKind
(int iHeap, int offset, SlotKinds.SlotKind kind, boolean storeIt) private int
private int
readDiff
(SlotKinds.SlotKind kind, int prev) private void
readDiffWithPrevTypeSlot
(SlotKinds.SlotKind kind, int iHeap, int offset, boolean storeIt) private long
private int
private void
readFsxPart
(IntVector fsIndexes) Each FS index is sorted, and output is by deltaprivate void
readHeader
(InputStream istream) private void
readHeapStoredArray
(int iHeap, boolean storeIt, Heap heapObj, int srcTypeCode) private void
private int
readIntoByteArray
(int length, boolean storeIt) private int
readIntoLongArray
(SlotKinds.SlotKind kind, int length, boolean storeIt) private int
readIntoShortArray
(int length, boolean storeIt) private long
readLongOrDouble
(SlotKinds.SlotKind kind, long prev) private void
readNonHeapStoredArray
(int iHeap, boolean storeIt) private int
readString
(boolean storeIt) private long
private long
readVlong
(DataInputStream dis) private int
S E R I A L I Z Eprivate int
serializeArrayLength
(int iHeap) private void
serializeByKind
(int iHeap, int offset) private void
serializeDiffWithPrevTypeSlot
(SlotKinds.SlotKind kind, int iHeap, int offset) private void
serializeHeapStoredArray
(int iHeap) private void
serializeNonHeapStoredArray
(int iHeap) private DataOutputStream
setupOutputStream
(int i) private void
setupOutputStreams
(Object out) Set up Streamsprivate void
setupReadStream
(int slotIndex, int bytesCompr, int bytesOrig) private void
(package private) static void
skipBytes
(DataInputStream stream, int skipNumber) private void
skipDouble
(int length) private void
skipLong
(int length) private int[]
private void
updatePrevIntValue
(int iHeap, int offset, int newValue) private void
write0
(int kind) private int
writeDiff
(int kind, int v, int prev) Encoding: bit 6 = sign: 1 = negative bit 7 = delta: 1 = deltaprivate void
writeDouble
(long raw) private void
writeFloat
(int raw) private void
writeFromByteArray
(SlotKinds.SlotKind kind, int startPos, int length) private void
writeFromLongArray
(SlotKinds.SlotKind kind, int startPos, int length) private void
writeFromShortArray
(int startPos, int length) private void
HEADERSprivate int
writeIntOrHeapRef
(int kind, int index, int prev) private void
writeLong
(long v, long prev) private long
writeLongFromHeapIndex
(int index, long prev) private void
private void
private void
writeUnsignedByte
(DataOutputStream s, int v) private void
writeVnumber
(int kind, int v) private void
writeVnumber
(int kind, long v) private void
writeVnumber
(DataOutputStream s, int v) private void
writeVnumber
(DataOutputStream s, long v)
-
Field Details
-
INT0
private static final int[] INT0 -
TRACE_SER
private static final boolean TRACE_SER- See Also:
-
TRACE_DES
private static final boolean TRACE_DES- See Also:
-
TRACE_STR_ARRAY
private static final boolean TRACE_STR_ARRAY- See Also:
-
VERSION
private static final int VERSIONVersion of the serializer/deserializer, used to allow deserialization of older versions Version 0 - initial SVN checkin Version 1 - changes to support CasTypeSystemMapper- See Also:
-
DBL_1
private static final long DBL_1 -
arrayLength_i
private static final int arrayLength_i -
heapRef_i
private static final int heapRef_i -
int_i
private static final int int_i -
byte_i
private static final int byte_i -
short_i
private static final int short_i -
typeCode_i
private static final int typeCode_i -
strOffset_i
private static final int strOffset_i -
strLength_i
private static final int strLength_i -
long_High_i
private static final int long_High_i -
long_Low_i
private static final int long_Low_i -
float_Mantissa_Sign_i
private static final int float_Mantissa_Sign_i -
float_Exponent_i
private static final int float_Exponent_i -
double_Mantissa_Sign_i
private static final int double_Mantissa_Sign_i -
double_Exponent_i
private static final int double_Exponent_i -
fsIndexes_i
private static final int fsIndexes_i -
strChars_i
private static final int strChars_i -
control_i
private static final int control_i -
strSeg_i
private static final int strSeg_i -
ts
Things set up for one instance of this class -
compressLevel
-
compressStrategy
-
isTypeMappingCmn
private boolean isTypeMappingCmnThings that are used by common routines among serialization and deserialization -
typeMapperCmn
-
cas
Things for both serialization and Deserialization -
heap
private int[] heap -
stringHeapObj
-
longHeapObj
-
shortHeapObj
-
byteHeapObj
-
heapStart
private int heapStart -
heapEnd
private int heapEnd -
totalMappedHeapSize
private int totalMappedHeapSize -
isSerializingDelta
private final boolean isSerializingDelta -
isDelta
private boolean isDelta -
isReadingDelta
private boolean isReadingDelta -
mark
-
fsStartIndexes
-
reuseInfoProvided
private final boolean reuseInfoProvided -
doMeasurements
private final boolean doMeasurements -
os
-
only1CommonString
private boolean only1CommonString -
tgtTs
-
typeInfo
-
typeMapper
-
isTypeMapping
private boolean isTypeMapping -
iPrevHeapArray
private final int[] iPrevHeapArray -
iPrevHeap
private int iPrevHeap -
prevHeapInstanceWithIntValues
private final int[][] prevHeapInstanceWithIntValuesHold prev instance of FS which have non-array FSRef slots, to allow computing these to match case where a 0 value is used because of type filtering for each typecode, only set if the type has 1 or more non-array fsref set only for non-filtered domain types set only for non-0 values if fsRef is to filtered type, value serialized will be 0, but this slot not set On deserialization: if value is 0, skip setting -
foundFSs
-
foundFSsBelowMark
-
foundFSsArray
private int[] foundFSsArray -
toBeScanned
-
debugEOF
private final boolean debugEOF- See Also:
-
serializedOut
Things for just serialization -
sm
-
baosZipSources
-
dosZipSources
-
savedAllIndexesFSs
private int[] savedAllIndexesFSs -
estimatedZipSize
private final int[] estimatedZipSize -
byte_dos
-
typeCode_dos
-
strOffset_dos
-
strLength_dos
-
float_Mantissa_Sign_dos
-
float_Exponent_dos
-
double_Mantissa_Sign_dos
-
double_Exponent_dos
-
fsIndexes_dos
-
control_dos
-
strSeg_dos
-
allowPreexistingFS
Things for just deserialization -
deserIn
-
version
private int version -
dataInputs
-
inflaters
-
fixupsNeeded
-
stringTableOffset
private int stringTableOffset -
longZeroIndex
private int longZeroIndexThese indexes remember sharable common values in aux heaps Values must be in aux heap, but not part of arrays there so that rules out boolean, byte, and shorts -
double1Index
private int double1Index -
isUpdatePrevOK
private boolean isUpdatePrevOK -
readCommonString
-
arrayLength_dis
-
heapRef_dis
-
int_dis
-
byte_dis
-
short_dis
-
typeCode_dis
-
strOffset_dis
-
strLength_dis
-
long_High_dis
-
long_Low_dis
-
float_Mantissa_Sign_dis
-
float_Exponent_dis
-
double_Mantissa_Sign_dis
-
double_Exponent_dis
-
fsIndexes_dis
-
strChars_dis
-
control_dis
-
strSeg_dis
-
-
Constructor Details
-
BinaryCasSerDes6
public BinaryCasSerDes6(AbstractCas aCas, MarkerImpl mark, TypeSystemImpl tgtTs, BinaryCasSerDes6.ReuseInfo rfs, boolean doMeasurements, BinaryCasSerDes6.CompressLevel compressLevel, BinaryCasSerDes6.CompressStrat compressStrategy) throws ResourceInitializationException Setup to serialize or deserialize using binary compression, with (optional) type mapping and only processing reachable Feature Structures- Parameters:
aCas
- required - refs the CAS being serialized or deserialized intomark
- if not null is the serialization mark for delta serialization. Unused for deserialization.tgtTs
- if not null is the target type system. For serialization - this is a subset of the CASs TSrfs
- For delta serialization - must be not null, and the saved value after deserializing the original before any modifications / additions made. For normal serialization - can be null, but if not, is used in place of re-calculating, for speed up For delta deserialization - must not be null, and is the saved value after serializing to the service For normal deserialization - must be nulldoMeasurements
- if true, measurements are done (on serialization)compressLevel
- if not null, specifies enum instance for compress levelcompressStrategy
- if not null, specifies enum instance for compress strategy- Throws:
ResourceInitializationException
- if the target type system is incompatible with the source type system
-
BinaryCasSerDes6
Setup to serialize (not delta) or deserialize (not delta) using binary compression, no type mapping but only processing reachable Feature Structures- Parameters:
cas
- -- Throws:
ResourceInitializationException
- never thrown
-
BinaryCasSerDes6
public BinaryCasSerDes6(AbstractCas cas, TypeSystemImpl tgtTs) throws ResourceInitializationException Setup to serialize (not delta) or deserialize (not delta) using binary compression, with type mapping and only processing reachable Feature Structures- Parameters:
cas
- -tgtTs
- -- Throws:
ResourceInitializationException
- if the target type system is incompatible with the source type system
-
BinaryCasSerDes6
public BinaryCasSerDes6(AbstractCas cas, MarkerImpl mark, TypeSystemImpl tgtTs, BinaryCasSerDes6.ReuseInfo rfs) throws ResourceInitializationException Setup to serialize (maybe delta) or deserialize (maybe delta) using binary compression, with type mapping and only processing reachable Feature Structures- Parameters:
cas
- -mark
- -tgtTs
- -rfs
- Reused Feature Structure information - required for both delta serialization and delta deserialization- Throws:
ResourceInitializationException
- if the target type system is incompatible with the source type system
-
BinaryCasSerDes6
public BinaryCasSerDes6(AbstractCas cas, MarkerImpl mark, TypeSystemImpl tgtTs, BinaryCasSerDes6.ReuseInfo rfs, boolean doMeasurements) throws ResourceInitializationException Setup to serialize (maybe delta) or deserialize (maybe delta) using binary compression, with type mapping and only processing reachable Feature Structures, output measurements- Parameters:
cas
- -mark
- -tgtTs
- -rfs
- Reused Feature Structure information - speed up on serialization, required on delta deserializationdoMeasurements
- -- Throws:
ResourceInitializationException
- if the target type system is incompatible with the source type system
-
BinaryCasSerDes6
public BinaryCasSerDes6(AbstractCas cas, BinaryCasSerDes6.ReuseInfo rfs) throws ResourceInitializationException Setup to serialize (not delta) or deserialize (maybe delta) using binary compression, no type mapping and only processing reachable Feature Structures- Parameters:
cas
- -rfs
- -- Throws:
ResourceInitializationException
- never thrown
-
-
Method Details
-
getReuseInfo
-
serialize
S E R I A L I Z E- Parameters:
out
- -- Returns:
- null or serialization measurements (depending on setting of doMeasurements)
- Throws:
IOException
- passthru
-
serializeHeapStoredArray
- Throws:
IOException
-
writeIntOrHeapRef
- Throws:
IOException
-
writeLongFromHeapIndex
- Throws:
IOException
-
serializeNonHeapStoredArray
- Throws:
IOException
-
serializeByKind
- Throws:
IOException
-
serializeArrayLength
- Throws:
IOException
-
serializeDiffWithPrevTypeSlot
private void serializeDiffWithPrevTypeSlot(SlotKinds.SlotKind kind, int iHeap, int offset) throws IOException - Throws:
IOException
-
updatePrevIntValue
private void updatePrevIntValue(int iHeap, int offset, int newValue) - Parameters:
iHeap
- index in the heapoffset
- offset to the slotnewValue
- for heap refs, is the converted-from-addr-to-seq-number value
-
initPrevIntValue
private int[] initPrevIntValue(int iHeap) -
getPrevIntValue
private int getPrevIntValue(int iHeap, int offset) - Parameters:
iHeap
- index in the heapoffset
- true offset, 1 = first feature...- Returns:
- the previous int value for use in difference calculations
-
collectAndZip
Method: write with deflation into a single byte array stream skip if not worth deflating skip the Slot_Control stream record in the Slot_Control stream, for each deflated stream: the Slot index the number of compressed bytes the number of uncompressed bytes add to header: nbr of compressed entries the Slot_Control stream size the Slot_Control stream all the zipped streams- Throws:
IOException
- passthru
-
writeLong
- Throws:
IOException
-
writeString
- Throws:
IOException
-
writeFloat
- Throws:
IOException
-
writeVnumber
- Throws:
IOException
-
writeVnumber
- Throws:
IOException
-
writeVnumber
- Throws:
IOException
-
writeVnumber
- Throws:
IOException
-
writeUnsignedByte
- Throws:
IOException
-
writeDouble
- Throws:
IOException
-
encodeIntSign
private int encodeIntSign(int v) -
writeDiff
Encoding: bit 6 = sign: 1 = negative bit 7 = delta: 1 = delta- Parameters:
kind
-i
- runs from iHeap + 3 to end of array sets isUpdatePrevOK true if ok to update prev, false if writing 0 for any reason, or max neg nbr- Throws:
IOException
- passthru
-
write0
- Throws:
IOException
-
writeFromByteArray
private void writeFromByteArray(SlotKinds.SlotKind kind, int startPos, int length) throws IOException - Throws:
IOException
-
writeFromLongArray
private void writeFromLongArray(SlotKinds.SlotKind kind, int startPos, int length) throws IOException - Throws:
IOException
-
writeFromShortArray
- Throws:
IOException
-
deserialize
- Parameters:
istream
- -- Throws:
IOException
- -
-
deserialize
public void deserialize(InputStream istream, AllowPreexistingFS allowPreexistingFS) throws IOException Version used by uima-as to read delta cas from remote parallel steps- Parameters:
istream
- input streamallowPreexistingFS
- what to do if item already exists below the mark- Throws:
IOException
- passthru
-
deserializeAfterVersion
public void deserializeAfterVersion(DataInputStream istream, boolean isDelta, AllowPreexistingFS allowPreexistingFS) throws IOException - Throws:
IOException
-
readNonHeapStoredArray
- Throws:
IOException
-
readArrayLength
- Throws:
IOException
-
readHeapStoredArray
private void readHeapStoredArray(int iHeap, boolean storeIt, Heap heapObj, int srcTypeCode) throws IOException - Throws:
IOException
-
readByKind
private void readByKind(int iHeap, int offset, SlotKinds.SlotKind kind, boolean storeIt) throws IOException - Parameters:
iHeap
- index in the heapoffset
- can be -1 - in which case read, but don't store- Throws:
IOException
- passthru
-
readIndexedFeatureStructures
- Throws:
IOException
-
readFsxPart
Each FS index is sorted, and output is by delta- Throws:
IOException
-
getInputStream
-
readVnumber
- Throws:
IOException
-
readVlong
- Throws:
IOException
-
readIntoByteArray
- Throws:
IOException
-
readIntoShortArray
- Throws:
IOException
-
readIntoLongArray
private int readIntoLongArray(SlotKinds.SlotKind kind, int length, boolean storeIt) throws IOException - Throws:
IOException
-
readDiffWithPrevTypeSlot
private void readDiffWithPrevTypeSlot(SlotKinds.SlotKind kind, int iHeap, int offset, boolean storeIt) throws IOException - Throws:
IOException
-
readDiff
- Throws:
IOException
-
readDiff
- Throws:
IOException
-
readLongOrDouble
- Throws:
IOException
-
skipLong
- Throws:
IOException
-
skipDouble
- Throws:
IOException
-
readFloat
- Throws:
IOException
-
decodeIntSign
private int decodeIntSign(int v) -
readDouble
- Throws:
IOException
-
decodeDouble
private long decodeDouble(long mants, int exponent) -
readVlong
- Throws:
IOException
-
readString
- Throws:
IOException
-
skipBytes
- Throws:
IOException
-
incrToNextFs
-
processIndexedFeatureStructures
- Throws:
IOException
-
processFsxPart
private int processFsxPart(int[] fsIndexes, int fsNdxStart, boolean isDoingEnqueue, boolean isWrite) throws IOException - Throws:
IOException
-
enqueueFS
private void enqueueFS(int fsAddr) -
isInstanceInTgtTs
private boolean isInstanceInTgtTs(int fsAddr) -
processRefedFSs
private void processRefedFSs() -
enqueueFeatures
private void enqueueFeatures(int addr) Enqueue all FSs reachable from features of the given FS. -
initFsStartIndexes
private int initFsStartIndexes()Serializing: Called at beginning of serialize, scans whole CAS or just delta CAS If doing delta serialization, fsStartIndexes is passed in, pre-initialized with a copy of the map info below the line.- Returns:
- amount of heap used in target, side effect: set up fsStartIndexes (for both src and tgt)
-
addStringFromFS
-
compareCASes
Compare 2 CASes, with perhaps different type systems. If the type systems are different, construct a type mapper and use that to selectively ignore types or features not in other type system The Mapper filters C1 -%gt; C2. Compare only feature structures reachable via indexes or refs The order must match- Parameters:
c1
- CAS to comparec2
- CAS to compare- Returns:
- true if equal (for types / features in both)
-
makeDataOutputStream
- Parameters:
f
- can be a DataOutputStream, an OutputStream a File- Returns:
- a data output stream
- Throws:
FileNotFoundException
- passthru
-
printCasInfo
-
setupOutputStreams
Set up Streams- Throws:
FileNotFoundException
- passthru
-
setupOutputStream
-
setupReadStreams
- Throws:
IOException
-
setupReadStream
- Throws:
IOException
-
closeDataInputs
private void closeDataInputs() -
writeHeader
HEADERS- Throws:
IOException
- passthru
-
readHeader
- Throws:
IOException
-
writeStringInfo
- Throws:
IOException
-
toArrayOrINT0
-