Package org.apache.lucene.tests.index

Class BaseKnnVectorsFormatTestCase

java.lang.Object
org.junit.Assert
org.apache.lucene.tests.util.LuceneTestCase
org.apache.lucene.tests.index.BaseIndexFileFormatTestCase
org.apache.lucene.tests.index.BaseKnnVectorsFormatTestCase
public abstract class BaseKnnVectorsFormatTestCase extends BaseIndexFileFormatTestCase
Base class aiming at testing vectors formats. To test a new format, all you need is to register a new Codec which uses it and extend this class and override BaseIndexFileFormatTestCase.getCodec().
WARNING: This API is experimental and might change in incompatible ways in the next release.

  • Constructor Details

    • BaseKnnVectorsFormatTestCase

      public BaseKnnVectorsFormatTestCase()

  • Method Details

    • init

      public void init()

    • supportsFloatVectorFallback

      protected abstract boolean supportsFloatVectorFallback()

    • getQuantizationBits

      protected int getQuantizationBits()
      Returns the number of bits used for quantization to compute epsilon tolerance of float quantization errors in test cases. Default is 8 bits, override in subclasses if needed

    • getCodecForFloatVectorFallbackTest

      protected Codec getCodecForFloatVectorFallbackTest()

    • addRandomFields

      protected void addRandomFields(Document doc)
      Description copied from class: BaseIndexFileFormatTestCase
      Add random fields to the provided document.
      Specified by:
      addRandomFields in class BaseIndexFileFormatTestCase

    • mergeIsStable

      protected boolean mergeIsStable()
      Overrides:
      mergeIsStable in class BaseIndexFileFormatTestCase

    • testFieldConstructor

      public void testFieldConstructor()

    • testFieldConstructorExceptions

      public void testFieldConstructorExceptions()

    • testFieldSetValue

      public void testFieldSetValue()

    • testIllegalDimChangeTwoDocs

      public void testIllegalDimChangeTwoDocs() throws Exception
      Throws:
      Exception

    • testIllegalSimilarityFunctionChange

      public void testIllegalSimilarityFunctionChange() throws Exception
      Throws:
      Exception

    • testIllegalDimChangeTwoWriters

      public void testIllegalDimChangeTwoWriters() throws Exception
      Throws:
      Exception

    • testMergingWithDifferentKnnFields

      public void testMergingWithDifferentKnnFields() throws Exception
      Throws:
      Exception

    • testMergingWithDifferentByteKnnFields

      public void testMergingWithDifferentByteKnnFields() throws Exception
      Throws:
      Exception

    • testWriterRamEstimate

      public void testWriterRamEstimate() throws Exception
      Throws:
      Exception

    • testIllegalSimilarityFunctionChangeTwoWriters

      public void testIllegalSimilarityFunctionChangeTwoWriters() throws Exception
      Throws:
      Exception

    • testAddIndexesDirectory0

      public void testAddIndexesDirectory0() throws Exception
      Throws:
      Exception

    • testAddIndexesDirectory1

      public void testAddIndexesDirectory1() throws Exception
      Throws:
      Exception

    • testAddIndexesDirectory01

      public void testAddIndexesDirectory01() throws Exception
      Throws:
      Exception

    • testIllegalDimChangeViaAddIndexesDirectory

      public void testIllegalDimChangeViaAddIndexesDirectory() throws Exception
      Throws:
      Exception

    • testIllegalSimilarityFunctionChangeViaAddIndexesDirectory

      public void testIllegalSimilarityFunctionChangeViaAddIndexesDirectory() throws Exception
      Throws:
      Exception

    • testIllegalDimChangeViaAddIndexesCodecReader

      public void testIllegalDimChangeViaAddIndexesCodecReader() throws Exception
      Throws:
      Exception

    • testIllegalSimilarityFunctionChangeViaAddIndexesCodecReader

      public void testIllegalSimilarityFunctionChangeViaAddIndexesCodecReader() throws Exception
      Throws:
      Exception

    • testIllegalDimChangeViaAddIndexesSlowCodecReader

      public void testIllegalDimChangeViaAddIndexesSlowCodecReader() throws Exception
      Throws:
      Exception

    • testIllegalSimilarityFunctionChangeViaAddIndexesSlowCodecReader

      public void testIllegalSimilarityFunctionChangeViaAddIndexesSlowCodecReader() throws Exception
      Throws:
      Exception

    • testIllegalMultipleValues

      public void testIllegalMultipleValues() throws Exception
      Throws:
      Exception

    • testIllegalDimensionTooLarge

      public void testIllegalDimensionTooLarge() throws Exception
      Throws:
      Exception

    • testIllegalEmptyVector

      public void testIllegalEmptyVector() throws Exception
      Throws:
      Exception

    • testDifferentCodecs1

      public void testDifferentCodecs1() throws Exception
      Throws:
      Exception

    • testDifferentCodecs2

      public void testDifferentCodecs2() throws Exception
      Throws:
      Exception

    • testInvalidKnnVectorFieldUsage

      public void testInvalidKnnVectorFieldUsage()

    • testDeleteAllVectorDocs

      public void testDeleteAllVectorDocs() throws Exception
      Throws:
      Exception

    • testKnnVectorFieldMissingFromOneSegment

      public void testKnnVectorFieldMissingFromOneSegment() throws Exception
      Throws:
      Exception

    • testSparseVectors

      public void testSparseVectors() throws Exception
      Throws:
      Exception

    • testFloatVectorScorerIteration

      public void testFloatVectorScorerIteration() throws Exception
      Throws:
      Exception

    • testByteVectorScorerIteration

      public void testByteVectorScorerIteration() throws Exception
      Throws:
      Exception

    • testEmptyFloatVectorData

      public void testEmptyFloatVectorData() throws Exception
      Throws:
      Exception

    • testEmptyByteVectorData

      public void testEmptyByteVectorData() throws Exception
      Throws:
      Exception

    • randomSimilarity

      protected VectorSimilarityFunction randomSimilarity()

    • randomVectorEncoding

      protected VectorEncoding randomVectorEncoding()
      This method is overrideable since old codec versions only support VectorEncoding.FLOAT32.

    • testIndexedValueNotAliased

      public void testIndexedValueNotAliased() throws Exception
      Throws:
      Exception

    • testSortedIndex

      public void testSortedIndex() throws Exception
      Throws:
      Exception

    • testSortedIndexBytes

      public void testSortedIndexBytes() throws Exception
      Throws:
      Exception

    • testIndexMultipleKnnVectorFields

      public void testIndexMultipleKnnVectorFields() throws Exception
      Throws:
      Exception

    • testRandom

      public void testRandom() throws Exception
      Index random vectors, sometimes skipping documents, sometimes deleting a document, sometimes merging, sometimes sorting the index, and verify that the expected values can be read back consistently.
      Throws:
      Exception

    • testRandomBytes

      public void testRandomBytes() throws Exception
      Index random vectors as bytes, sometimes skipping documents, sometimes deleting a document, sometimes merging, sometimes sorting the index, and verify that the expected values can be read back consistently.
      Throws:
      Exception

    • testSearchWithVisitedLimit

      public void testSearchWithVisitedLimit() throws Exception
      Tests whether KnnVectorsReader.search(java.lang.String, float[], org.apache.lucene.search.KnnCollector, org.apache.lucene.search.AcceptDocs) implementations obey the limit on the number of visited vectors. This test is a best-effort attempt to capture the right behavior, and isn't meant to define a strict requirement on behavior.
      Throws:
      Exception

    • testRandomWithUpdatesAndGraph

      public void testRandomWithUpdatesAndGraph() throws Exception
      Index random vectors, sometimes skipping documents, sometimes updating a document, sometimes merging, sometimes sorting the index, using an HNSW similarity function to also produce a graph, and verify that the expected values can be read back consistently.
      Throws:
      Exception

    • randomVector

      public static float[] randomVector(int dim)

    • randomNormalizedVector

      public static float[] randomNormalizedVector(int dim)

    • randomVector8

      public static byte[] randomVector8(int dim)

    • testCheckIndexIncludesVectors

      public void testCheckIndexIncludesVectors() throws Exception
      Throws:
      Exception

    • testSimilarityFunctionIdentifiers

      public void testSimilarityFunctionIdentifiers()

    • testVectorEncodingOrdinals

      public void testVectorEncodingOrdinals()

    • testAdvance

      public void testAdvance() throws Exception
      Throws:
      Exception

    • testVectorValuesReportCorrectDocs

      public void testVectorValuesReportCorrectDocs() throws Exception
      Throws:
      Exception

    • testReadQuantizedVectorWithEmptyRawVectors

      public void testReadQuantizedVectorWithEmptyRawVectors() throws Exception
      Tests reading quantized vectors when raw vector data is empty. Verifies that scalar quantized formats can properly dequantize vectors and maintain accuracy within expected error bounds even when the original raw vector file is empty or corrupted.
      Throws:
      Exception

    • simulateEmptyRawVectors

      protected void simulateEmptyRawVectors(Directory dir) throws Exception
      Simulates empty raw vectors by modifying index files. Override in codecs that support FloatVector fallback.
      Throws:
      Exception

    • testMismatchedFields

      public void testMismatchedFields() throws Exception
      Throws:
      Exception

    • assertOffHeapByteSize

      protected void assertOffHeapByteSize(LeafReader r, String fieldName) throws IOException
      Throws:
      IOException

    • testMergeOffHeapByteSizeMaps

      public void testMergeOffHeapByteSizeMaps()