Class ReftableCompactor


  • public class ReftableCompactor
    extends java.lang.Object
    Merges reftables and compacts them into a single output.

    For a partial compaction callers should setIncludeDeletes(boolean) to true to ensure the new reftable continues to use a delete marker to shadow any lower reftable that may have the reference present.

    By default all log entries within the range defined by setReflogExpireMinUpdateIndex(long) and setReflogExpireMaxUpdateIndex(long) are copied, even if no references in the output file match the log records. Callers may truncate the log to a more recent time horizon with setReflogExpireOldestReflogTimeMillis(long), or disable the log altogether with setOldestReflogTimeMillis(Long.MAX_VALUE).

    • Field Detail

      • includeDeletes

        private boolean includeDeletes
      • reflogExpireMinUpdateIndex

        private long reflogExpireMinUpdateIndex
      • reflogExpireMaxUpdateIndex

        private long reflogExpireMaxUpdateIndex
      • reflogExpireOldestReflogTimeMillis

        private long reflogExpireOldestReflogTimeMillis
    • Constructor Detail

      • ReftableCompactor

        public ReftableCompactor​(java.io.OutputStream out)
        Creates a new compactor.
        Parameters:
        out - stream to write the compacted tables to. Caller is responsible for closing out.
    • Method Detail

      • setConfig

        public ReftableCompactor setConfig​(ReftableConfig cfg)
        Set configuration for the reftable.
        Parameters:
        cfg - configuration for the reftable.
        Returns:
        this
      • setIncludeDeletes

        public ReftableCompactor setIncludeDeletes​(boolean deletes)
        Whether to include deletions in the output, which may be necessary for partial compaction.
        Parameters:
        deletes - true to include deletions in the output, which may be necessary for partial compaction.
        Returns:
        this
      • setReflogExpireMinUpdateIndex

        public ReftableCompactor setReflogExpireMinUpdateIndex​(long min)
        Set the minimum update index for log entries that appear in the compacted reftable.
        Parameters:
        min - the minimum update index for log entries that appear in the compacted reftable. This should be 1 higher than the prior reftable's maxUpdateIndex if this table will be used in a stack.
        Returns:
        this
      • setReflogExpireMaxUpdateIndex

        public ReftableCompactor setReflogExpireMaxUpdateIndex​(long max)
        Set the maximum update index for log entries that appear in the compacted reftable.
        Parameters:
        max - the maximum update index for log entries that appear in the compacted reftable. This should be at least 1 higher than the prior reftable's maxUpdateIndex if this table will be used in a stack.
        Returns:
        this
      • setReflogExpireOldestReflogTimeMillis

        public ReftableCompactor setReflogExpireOldestReflogTimeMillis​(long timeMillis)
        Set oldest reflog time to preserve.
        Parameters:
        timeMillis - oldest log time to preserve. Entries whose timestamps are >= timeMillis will be copied into the output file. Log entries that predate timeMillis will be discarded. Specified in Java standard milliseconds since the epoch.
        Returns:
        this
      • addAll

        public void addAll​(java.util.List<ReftableReader> readers)
                    throws java.io.IOException
        Add all of the tables, in the specified order.
        Parameters:
        readers - tables to compact. Tables should be ordered oldest first/most recent last so that the more recent tables can shadow the older results. Caller is responsible for closing the readers.
        Throws:
        java.io.IOException - update indexes of a reader cannot be accessed.
      • compact

        public void compact()
                     throws java.io.IOException
        Write a compaction to out.
        Throws:
        java.io.IOException - if tables cannot be read, or cannot be written.
      • getStats

        public ReftableWriter.Stats getStats()
        Get statistics of the last written reftable.
        Returns:
        statistics of the last written reftable.
      • mergeRefs

        private void mergeRefs​(MergedReftable mr)
                        throws java.io.IOException
        Throws:
        java.io.IOException
      • mergeLogs

        private void mergeLogs​(MergedReftable mr)
                        throws java.io.IOException
        Throws:
        java.io.IOException