Class CompressingStoredFieldsWriter

    • Constructor Detail

      • CompressingStoredFieldsWriter

        public CompressingStoredFieldsWriter​(Directory directory,
                                             SegmentInfo si,
                                             java.lang.String segmentSuffix,
                                             IOContext context,
                                             java.lang.String formatName,
                                             CompressionMode compressionMode,
                                             int chunkSize,
                                             int maxDocsPerChunk,
                                             int blockSize)
                                      throws java.io.IOException
        Sole constructor.
        Throws:
        java.io.IOException
    • Method Detail

      • close

        public void close()
                   throws java.io.IOException
        Specified by:
        close in interface java.lang.AutoCloseable
        Specified by:
        close in interface java.io.Closeable
        Specified by:
        close in class StoredFieldsWriter
        Throws:
        java.io.IOException
      • finishDocument

        public void finishDocument()
                            throws java.io.IOException
        Description copied from class: StoredFieldsWriter
        Called when a document and all its fields have been added.
        Overrides:
        finishDocument in class StoredFieldsWriter
        Throws:
        java.io.IOException
      • saveInts

        private static void saveInts​(int[] values,
                                     int length,
                                     DataOutput out)
                              throws java.io.IOException
        Throws:
        java.io.IOException
      • writeHeader

        private void writeHeader​(int docBase,
                                 int numBufferedDocs,
                                 int[] numStoredFields,
                                 int[] lengths,
                                 boolean sliced)
                          throws java.io.IOException
        Throws:
        java.io.IOException
      • triggerFlush

        private boolean triggerFlush()
      • flush

        private void flush()
                    throws java.io.IOException
        Throws:
        java.io.IOException
      • writeZFloat

        static void writeZFloat​(DataOutput out,
                                float f)
                         throws java.io.IOException
        Writes a float in a variable-length format. Writes between one and five bytes. Small integral values typically take fewer bytes.

        ZFloat --> Header, Bytes*?

        • Header --> Uint8. When it is equal to 0xFF then the value is negative and stored in the next 4 bytes. Otherwise if the first bit is set then the other bits in the header encode the value plus one and no other bytes are read. Otherwise, the value is a positive float value whose first byte is the header, and 3 bytes need to be read to complete it.
        • Bytes --> Potential additional bytes to read depending on the header.
        Throws:
        java.io.IOException
      • writeZDouble

        static void writeZDouble​(DataOutput out,
                                 double d)
                          throws java.io.IOException
        Writes a float in a variable-length format. Writes between one and five bytes. Small integral values typically take fewer bytes.

        ZFloat --> Header, Bytes*?

        • Header --> Uint8. When it is equal to 0xFF then the value is negative and stored in the next 8 bytes. When it is equal to 0xFE then the value is stored as a float in the next 4 bytes. Otherwise if the first bit is set then the other bits in the header encode the value plus one and no other bytes are read. Otherwise, the value is a positive float value whose first byte is the header, and 7 bytes need to be read to complete it.
        • Bytes --> Potential additional bytes to read depending on the header.
        Throws:
        java.io.IOException
      • writeTLong

        static void writeTLong​(DataOutput out,
                               long l)
                        throws java.io.IOException
        Writes a long in a variable-length format. Writes between one and ten bytes. Small values or values representing timestamps with day, hour or second precision typically require fewer bytes.

        ZLong --> Header, Bytes*?

        • Header --> The first two bits indicate the compression scheme:
          • 00 - uncompressed
          • 01 - multiple of 1000 (second)
          • 10 - multiple of 3600000 (hour)
          • 11 - multiple of 86400000 (day)
          Then the next bit is a continuation bit, indicating whether more bytes need to be read, and the last 5 bits are the lower bits of the encoded value. In order to reconstruct the value, you need to combine the 5 lower bits of the header with a vLong in the next bytes (if the continuation bit is set to 1). Then zigzag-decode it and finally multiply by the multiple corresponding to the compression scheme.
        • Bytes --> Potential additional bytes to read depending on the header.
        Throws:
        java.io.IOException
      • tooDirty

        boolean tooDirty​(CompressingStoredFieldsReader candidate)
        Returns true if we should recompress this reader, even though we could bulk merge compressed data

        The last chunk written for a segment is typically incomplete, so without recompressing, in some worst-case situations (e.g. frequent reopen with tiny flushes), over time the compression ratio can degrade. This is a safety switch.