Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
90 changes: 41 additions & 49 deletions entity-framework/core/providers/cosmos/full-text-search.md
Original file line number Diff line number Diff line change
Expand Up @@ -46,34 +46,34 @@ public class BloggingContext
Full-text search operations are language specific, using American English (`en-US`) by default. You can customize the language for individual properties as part of `EnableFullTextSearch` call:

```c#
protected override void OnModelCreating(ModelBuilder modelBuilder)
protected override void OnModelCreating(ModelBuilder modelBuilder)
{
modelBuilder.Entity<Blog>(b =>
{
modelBuilder.Entity<Blog>(b =>
{
b.Property(x => x.Contents).EnableFullTextSearch();
b.HasIndex(x => x.Contents).IsFullTextIndex();
b.Property(x => x.ContentsGerman).EnableFullTextSearch("de-DE");
b.HasIndex(x => x.ContentsGerman).IsFullTextIndex();
});
}
b.Property(x => x.Contents).EnableFullTextSearch();
b.HasIndex(x => x.Contents).IsFullTextIndex();
b.Property(x => x.ContentsGerman).EnableFullTextSearch("de-DE");
b.HasIndex(x => x.ContentsGerman).IsFullTextIndex();
});
}
```

You can also set a default language for the container - unless overridden in the `EnableFullTextSearch` method, all full-text properties inside the container will use that language.

```c#
protected override void OnModelCreating(ModelBuilder modelBuilder)
protected override void OnModelCreating(ModelBuilder modelBuilder)
{
modelBuilder.Entity<Blog>(b =>
{
modelBuilder.Entity<Blog>(b =>
{
b.HasDefaultFullTextLanguage("de-DE");
b.Property(x => x.ContentsEnglish).EnableFullTextSearch("en-US");
b.HasIndex(x => x.ContentsEnglish).IsFullTextIndex();
b.Property(x => x.ContentsGerman).EnableFullTextSearch();
b.HasIndex(x => x.ContentsGerman).IsFullTextIndex();
b.Property(x => x.TagsGerman).EnableFullTextSearch();
b.HasIndex(x => x.TagsGerman).IsFullTextIndex();
});
}
b.HasDefaultFullTextLanguage("de-DE");
b.Property(x => x.ContentsEnglish).EnableFullTextSearch("en-US");
b.HasIndex(x => x.ContentsEnglish).IsFullTextIndex();
b.Property(x => x.ContentsGerman).EnableFullTextSearch();
b.HasIndex(x => x.ContentsGerman).IsFullTextIndex();
b.Property(x => x.TagsGerman).EnableFullTextSearch();
b.HasIndex(x => x.TagsGerman).IsFullTextIndex();
});
}
```

## Querying
Expand All @@ -94,41 +94,33 @@ var mostInteresting = await context.Blogs.OrderBy(x => EF.Functions.FullTextScor

## Hybrid search

Full-text search can be used with vector search in the same query (i.e. hybrid search), by combining results of `FullTextScore` and `VectorDistance` functions. It can be done using the [`RRF`](/azure/cosmos-db/nosql/query/rrf) (Reciprocal Rank Fusion) function, which EF Core also provides inside `EF.Functions`:
Full-text search can be used with [vector search](xref:core/providers/cosmos/vector-search) in the same query; this is sometimes known as "hybrid search", and involves combining the scoring results from multiple searches via the [RRF (Reciprocal Rank Fusion) function](/azure/cosmos-db/nosql/query/rrf). Once you have your vector and full-text search configuration properly set up, you can perform hybrid search as follows:

```c#
public class Blog
{
...

public float[] Vector { get; set; }
public string Contents { get; set; }
}

public class BloggingContext
{
...

protected override void OnModelCreating(ModelBuilder modelBuilder)
{
modelBuilder.Entity<Blog>(b =>
{
b.Property(x => x.Contents).EnableFullTextSearch();
b.HasIndex(x => x.Contents).IsFullTextIndex();

b.Property(x => x.Vector).IsVectorProperty(DistanceFunction.Cosine, dimensions: 1536);
b.HasIndex(x => x.Vector).IsVectorIndex(VectorIndexType.Flat);
});
}
}

float[] myVector = /* generate vector data from text, image, etc. */
var hybrid = await context.Blogs.OrderBy(x => EF.Functions.Rrf(
var hybrid = await context.Blogs
.OrderBy(x => EF.Functions.Rrf(
EF.Functions.FullTextScore(x.Contents, "database"),
EF.Functions.VectorDistance(x.Vector, myVector)))
.Take(10)
.ToListAsync();
```

The RRF function also allows assigning different weights to each search function, allowing e.g. the vector search to have great weight in the overall results:

```c#
float[] myVector = /* generate vector data from text, image, etc. */
var hybrid = await context.Blogs
.OrderBy(x => EF.Functions.Rrf(
new[]
{
EF.Functions.FullTextScore(x.Contents, "database"),
EF.Functions.VectorDistance(x.Vector, myVector)
},
weights: new[] { 1, 2 }))
.Take(10)
.ToListAsync();
```

> [!TIP]
> You can combine more than two scoring functions inside `Rrf` call, as well as using only `FullTextScore`, or only `VectorDistance`.
> You can combine more than two scoring functions inside `Rrf` call, as well as using only `FullTextScore`, or only `VectorDistance` invocations.
59 changes: 35 additions & 24 deletions entity-framework/core/providers/cosmos/querying.md
Original file line number Diff line number Diff line change
Expand Up @@ -223,19 +223,19 @@ This section shows which .NET methods and members are translated into which SQL
------------------------------------------ | --------------------------------------------------------------------------------- | --------
DateTime.UtcNow | [GetCurrentDateTime()](/azure/cosmos-db/nosql/query/getcurrentdatetime)
DateTimeOffset.UtcNow | [GetCurrentDateTime()](/azure/cosmos-db/nosql/query/getcurrentdatetime)
dateTime.Year<sup>1</sup> | [DateTimePart("yyyy", dateTime)](/azure/cosmos-db/nosql/query/datetimepart) | EF Core 9.0
dateTimeOffset.Year<sup>1</sup> | [DateTimePart("yyyy", dateTimeOffset)](/azure/cosmos-db/nosql/query/datetimepart) | EF Core 9.0
dateTime.AddYears(years)<sup>1</sup> | [DateTimeAdd("yyyy", dateTime)](/azure/cosmos-db/nosql/query/datetimeadd) | EF Core 9.0
dateTimeOffset.AddYears(years)<sup>1</sup> | [DateTimeAdd("yyyy", dateTimeOffset)](/azure/cosmos-db/nosql/query/datetimeadd) | EF Core 9.0
dateTime.Year<sup>1</sup> | [DateTimePart("yyyy", dateTime)](/azure/cosmos-db/nosql/query/datetimepart) | EF 9
dateTimeOffset.Year<sup>1</sup> | [DateTimePart("yyyy", dateTimeOffset)](/azure/cosmos-db/nosql/query/datetimepart) | EF 9
dateTime.AddYears(years)<sup>1</sup> | [DateTimeAdd("yyyy", dateTime)](/azure/cosmos-db/nosql/query/datetimeadd) | EF 9
dateTimeOffset.AddYears(years)<sup>1</sup> | [DateTimeAdd("yyyy", dateTimeOffset)](/azure/cosmos-db/nosql/query/datetimeadd) | EF 9

<sup>1</sup> The other component members are translated as well (Month, Day...).

### Numeric functions

.NET | SQL | Added in
-------------------------- | --------------------------------------------------- | --------
double.DegreesToRadians(x) | [RADIANS(@x)](/azure/cosmos-db/nosql/query/radians) | EF Core 8.0
double.RadiansToDegrees(x) | [DEGREES(@x)](/azure/cosmos-db/nosql/query/degrees) | EF Core 8.0
.NET | SQL
-------------------------- | ---------------------------------------------------
double.DegreesToRadians(x) | [RADIANS(@x)](/azure/cosmos-db/nosql/query/radians)
double.RadiansToDegrees(x) | [DEGREES(@x)](/azure/cosmos-db/nosql/query/degrees)
EF.Functions.Random() | [RAND()](/azure/cosmos-db/nosql/query/rand)
Math.Abs(value) | [ABS(@value)](/azure/cosmos-db/nosql/query/abs)
Math.Acos(d) | [ACOS(@d)](/azure/cosmos-db/nosql/query/acos)
Expand Down Expand Up @@ -266,17 +266,17 @@ Math.Truncate(d) | [TRUNC(@d)](/azure/cosmos-db/nosql/query/trunc)

.NET | SQL | Added in
----------------------------------------------------------------- | ---------------------------------------------------------------------------------- | --------
Regex.IsMatch(input, pattern) | [RegexMatch(@pattern, @input)](/azure/cosmos-db/nosql/query/regexmatch) | EF Core 7.0
Regex.IsMatch(input, pattern, options) | [RegexMatch(@input, @pattern, @options)](/azure/cosmos-db/nosql/query/regexmatch) | EF Core 7.0
Regex.IsMatch(input, pattern) | [RegexMatch(@pattern, @input)](/azure/cosmos-db/nosql/query/regexmatch)
Regex.IsMatch(input, pattern, options) | [RegexMatch(@input, @pattern, @options)](/azure/cosmos-db/nosql/query/regexmatch)
string.Concat(str0, str1) | @str0 + @str1
string.Equals(a, b, StringComparison.Ordinal) | [STRINGEQUALS(@a, @b)](/azure/cosmos-db/nosql/query/stringequals)
string.Equals(a, b, StringComparison.OrdinalIgnoreCase) | [STRINGEQUALS(@a, @b, true)](/azure/cosmos-db/nosql/query/stringequals)
stringValue.Contains(value) | [CONTAINS(@stringValue, @value)](/azure/cosmos-db/nosql/query/contains)
stringValue.Contains(value, StringComparison.Ordinal) | [CONTAINS(@stringValue, @value, false)](/azure/cosmos-db/nosql/query/contains) | EF Core 9.0
stringValue.Contains(value, StringComparison.OrdinalIgnoreCase) | [CONTAINS(@stringValue, @value, true)](/azure/cosmos-db/nosql/query/contains) | EF Core 9.0
stringValue.Contains(value, StringComparison.Ordinal) | [CONTAINS(@stringValue, @value, false)](/azure/cosmos-db/nosql/query/contains) | EF 9
stringValue.Contains(value, StringComparison.OrdinalIgnoreCase) | [CONTAINS(@stringValue, @value, true)](/azure/cosmos-db/nosql/query/contains) | EF 9
stringValue.EndsWith(value) | [ENDSWITH(@stringValue, @value)](/azure/cosmos-db/nosql/query/endswith)
stringValue.EndsWith(value, StringComparison.Ordinal) | [ENDSWITH(@stringValue, @value, false)](/azure/cosmos-db/nosql/query/endswith) | EF Core 9.0
stringValue.EndsWith(value, StringComparison.OrdinalIgnoreCase) | [ENDSWITH(@stringValue, @value, true)](/azure/cosmos-db/nosql/query/endswith) | EF Core 9.0
stringValue.EndsWith(value, StringComparison.Ordinal) | [ENDSWITH(@stringValue, @value, false)](/azure/cosmos-db/nosql/query/endswith) | EF 9
stringValue.EndsWith(value, StringComparison.OrdinalIgnoreCase) | [ENDSWITH(@stringValue, @value, true)](/azure/cosmos-db/nosql/query/endswith) | EF 9
stringValue.Equals(value, StringComparison.Ordinal) | [STRINGEQUALS(@stringValue, @value)](/azure/cosmos-db/nosql/query/stringequals)
stringValue.Equals(value, StringComparison.OrdinalIgnoreCase) | [STRINGEQUALS(@stringValue, @value, true)](/azure/cosmos-db/nosql/query/stringequals)
stringValue.FirstOrDefault() | [LEFT(@stringValue, 1)](/azure/cosmos-db/nosql/query/left)
Expand All @@ -286,8 +286,8 @@ stringValue.LastOrDefault() | [RIGHT(@stri
stringValue.Length | [LENGTH(@stringValue)](/azure/cosmos-db/nosql/query/length)
stringValue.Replace(oldValue, newValue) | [REPLACE(@stringValue, @oldValue, @newValue)](/azure/cosmos-db/nosql/query/replace)
stringValue.StartsWith(value) | [STARTSWITH(@stringValue, @value)](/azure/cosmos-db/nosql/query/startswith)
stringValue.StartsWith(value, StringComparison.Ordinal) | [STARTSWITH(@stringValue, @value, false)](/azure/cosmos-db/nosql/query/startswith) | EF Core 9.0
stringValue.StartsWith(value, StringComparison.OrdinalIgnoreCase) | [STARTSWITH(@stringValue, @value, true)](/azure/cosmos-db/nosql/query/startswith) | EF Core 9.0
stringValue.StartsWith(value, StringComparison.Ordinal) | [STARTSWITH(@stringValue, @value, false)](/azure/cosmos-db/nosql/query/startswith) | EF 9
stringValue.StartsWith(value, StringComparison.OrdinalIgnoreCase) | [STARTSWITH(@stringValue, @value, true)](/azure/cosmos-db/nosql/query/startswith) | EF 9
stringValue.Substring(startIndex) | [SUBSTRING(@stringValue, @startIndex, LENGTH(@stringValue))](/azure/cosmos-db/nosql/query/substring)
stringValue.Substring(startIndex, length) | [SUBSTRING(@stringValue, @startIndex, @length)](/azure/cosmos-db/nosql/query/substring)
stringValue.ToLower() | [LOWER(@stringValue)](/azure/cosmos-db/nosql/query/lower)
Expand All @@ -296,17 +296,28 @@ stringValue.Trim() | [TRIM(@strin
stringValue.TrimEnd() | [RTRIM(@stringValue)](/azure/cosmos-db/nosql/query/rtrim)
stringValue.TrimStart() | [LTRIM(@stringValue)](/azure/cosmos-db/nosql/query/ltrim)

### Vector and full-text search

.NET | SQL | Added in
--------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------- | -----
EF.Functions.VectorDistance(vector1, vector2). | [VectorDistance(vector1, vector2)](/azure/cosmos-db/nosql/query/vectordistance) | EF 9
EF.Functions.VectorDistance(vector1, vector2, bruteForce) | [VectorDistance(vector1, vector2, bruteForce)](/azure/cosmos-db/nosql/query/vectordistance) | EF 9
EF.Functions.VectorDistance(vector1, vector2, bruteForce, distanceFunction) | [VectorDistance(vector1, vector2, bruteForce, distanceFunction)](/azure/cosmos-db/nosql/query/vectordistance) | EF 9
EF.Functions.FullTextContains(property, keyword) | [FullTextContains(property, keyword)](/azure/cosmos-db/nosql/query/fulltextcontains) | EF 10
EF.Functions.FullTextContainsAll(property, keyword1, keyword2) | [FullTextContainsAll(property, keyword1, keyword2)](/azure/cosmos-db/nosql/query/fulltextcontainsall) | EF 10
EF.Functions.FullTextContainsAny(property, keyword1, keyword2) | [FullTextContainsAny(property, keyword1, keyword2)](/azure/cosmos-db/nosql/query/fulltextcontainsany) | EF 10
EF.Functions.FullTextScore(property, keyword1, keyword2) | [FullTextScore(property, keyword1, keyword2)](/azure/cosmos-db/nosql/query/fulltextscore) | EF 10
EF.Functions.Rrf(search1, search2) | [RRF(property, search1, search2)](/azure/cosmos-db/nosql/query/rrf). | EF 10
EF.Functions.Rrf(new[] { search1, search2 }, weights) | [RRF(property, search1, search2, weights)](/azure/cosmos-db/nosql/query/rrf) | EF 10

For more information on vector search, see [the documentation](xref:core/providers/cosmos/vector-search). For more information on full-text search, see [the documentation](xref:core/providers/cosmos/full-text-search).

### Miscellaneous functions

.NET | SQL | Notes
.NET | SQL | Added in
--------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------- | -----
collection.Contains(item) | @item IN @collection
EF.Functions.CoalesceUndefined(x, y)<sup>1</sup> | [x ?? y](/azure/cosmos-db/nosql/query/ternary-coalesce-operators#coalesce-operator) | Added in EF Core 9.0
EF.Functions.IsDefined(x) | [IS_DEFINED(x)](/azure/cosmos-db/nosql/query/is-defined) | Added in EF Core 9.0
EF.Functions.VectorDistance(vector1, vector2)<sup>2</sup> | [VectorDistance(vector1, vector2)](/azure/cosmos-db/nosql/query/vectordistance) | Added in EF Core 9.0, Experimental
EF.Functions.VectorDistance(vector1, vector2, bruteForce)<sup>2</sup> | [VectorDistance(vector1, vector2, bruteForce)](/azure/cosmos-db/nosql/query/vectordistance) | Added in EF Core 9.0, Experimental
EF.Functions.VectorDistance(vector1, vector2, bruteForce, distanceFunction)<sup>2</sup> | [VectorDistance(vector1, vector2, bruteForce, distanceFunction)](/azure/cosmos-db/nosql/query/vectordistance) | Added in EF Core 9.0, Experimental
EF.Functions.CoalesceUndefined(x, y)<sup>1</sup> | [x ?? y](/azure/cosmos-db/nosql/query/ternary-coalesce-operators#coalesce-operator) | EF 9
EF.Functions.IsDefined(x) | [IS_DEFINED(x)](/azure/cosmos-db/nosql/query/is-defined) | EF 9

<sup>1</sup> Note that `EF.Functions.CoalesceUndefined` coalesces `undefined`, not `null`. To coalesce `null`, use the regular C# `??` operator.

<sup>2</sup> [See the documentation](xref:core/providers/cosmos/vector-search) for information on using vector search in Azure Cosmos DB, which is experimental. The APIs are subject to change.
Loading