How to do spatial filtering in Umbraco 10+
# help-with-umbraco
d
I've been tinkering with lucene indexes and managed to get a facetted search working using lucene directly and circumventing examine. Now I need to do a spatial search and sort as well, but I find myself somewhat stuck. Every time it kinda works, but it never seems to work good enough. I was hoping somebody might have some experience with this stuff and could share some tips and tricks. I'm using this example on github: https://gist.github.com/Mark-Broadhurst/8931898. The sorting seems to work, but the filtering is off. Taking two locations that are 33km apart, the filter already excludes them at 54km. 54km happens to be approximately equal to 33 miles, but I only use the distance in this part of the code:
DistanceUtils.Dist2Degrees(distance, DistanceUtils.EARTH_MEAN_RADIUS_KM)
and I verified that the mean radius in kilometers is correct
s
Just to add this older one here as well, not sure if you found it: https://shazwazza.com/post/spatial-search-with-examine-and-lucene/ And then @Mike Chambers seems to have a lot of experience as well 🙂
b
The article from Shannon helped me in a recent project, but it seems it isn't mentioned what the field
GeoLocationFieldName
is... I guess it was a field set in Lucene document writing event using same strategy as when searching.
In a component add this:
Copy code
public void Initialize()
{
    if (!_examineManager.TryGetIndex("ExternalIndex", out IIndex cindex))
    {
        throw new InvalidOperationException($"No index found by name ExternalIndex");
    }

    if (cindex is not LuceneIndex luceneIndex)
    {
        throw new InvalidOperationException($"Index ExternalIndex is not a LuceneIndex.");
    }

    SpatialContext ctx = SpatialContext.Geo;
    int maxLevels = 11; //results in sub-meter precision for geohash
    SpatialPrefixTree grid = new GeohashPrefixTree(ctx, maxLevels);
    var strategy = new RecursivePrefixTreeStrategy(grid, Constants.Examine.CourseInstance.FieldNames.GeoLocation);

    luceneIndex.DocumentWriting += (sender, args) => LuceneIndex_DocumentWriting(args, ctx, strategy);
}

private void LuceneIndex_DocumentWriting(DocumentWritingEventArgs e, SpatialContext ctx, SpatialStrategy strategy)
{
    if (e.Document.GetField("latitude") == null ||
        e.Document.GetField("longitude") == null)
        return;

    double latitude = double.Parse(e.ValueSet.Values["latitude"].First().ToString() ?? string.Empty, CultureInfo.InvariantCulture);
    double longitude = double.Parse(e.ValueSet.Values["longitude"].First().ToString() ?? string.Empty, CultureInfo.InvariantCulture);

    GetXYFromCoords(latitude, longitude, out var x, out var y);
    IPoint geoPoint = ctx.MakePoint(x, y);

    foreach (Field field in strategy.CreateIndexableFields(geoPoint))
    {
        e.Document.Add(field);
    }

    // I don't think this stored field is necessary to use spartial search, but is shown in Examine dashboard.
    e.Document.Add(new StoredField(strategy.FieldName, Invariant($"{geoPoint.X} {geoPoint.Y}")));
}

private void GetXYFromCoords(double lat, double lng, out double x, out double y)
{
    // Important! we need to change to x/y coords, longitude = x, latitude = y
    x = lng;
    y = lat;
}
https://gist.github.com/bjarnef/6b427123903c90e034027bb8a3a5d43a
c
Lars-Erik helped me with this recently
Copy code
cs
        var index = (LuceneIndex)_examineManager.GetIndex(Constants.ExamineIndexes.WMNIndex);
        var query = (LuceneSearchQueryBase)index.Searcher.CreateQuery(null, BooleanOperation.Or);
        var searcher = new IndexSearcher(index.IndexWriter.IndexWriter.GetReader(false));
        var filteredQuery = index
            .Searcher
            .CreateQuery()
            .NativeQuery($"+__IndexType:{IndexTypes.Content}")
            .And()
            .GroupedOr(new[] { "__NodeTypeAlias" }, searchModel.ContentTypes.ToArray());

        if (!string.IsNullOrWhiteSpace(searchModel?.SearchQuery?.Phrase ?? ""))
        {
            filteredQuery
                .And()
                .GroupedAnd(new[] { "name" }, searchModel.SearchQuery.Terms);
        }

        if (searchModel?.FacetSets != null && searchModel.FacetSets.Any())
        {
            foreach (var facetSet in searchModel.FacetSets)
            {
                if (facetSet.SelectedValues != null && facetSet.SelectedValues.Any())
                {
                    filteredQuery.And()
                    .GroupedOr(new[] { facetSet.PropertyAlias }, facetSet.SelectedValues);
                }
            }
        }
Copy code
cs
        Match match = Regex.Match(filteredQuery.ToString(), @"LuceneQuery:(.*?)\}");
        if (match.Success)
        {
            string luceneQuery = match.Groups[1].Value.Trim();
            var filterLuceneQuery = query.QueryParser.Parse(luceneQuery);
            query.Query.Add(filterLuceneQuery, Occur.MUST);
        }

        var field = new SortField("score", SortFieldType.SCORE, true);
        var sort = new Sort(field);

        var hasLocationPoint = !string.IsNullOrWhiteSpace(searchModel.Longitude) && !string.IsNullOrWhiteSpace(searchModel.Latitude);
        if (hasLocationPoint)
        {
            BooleanQuery geoQuery = null;

            var valueType = (ShapeFieldValueType)index.FieldValueTypeCollection.ValueTypes
                .FirstOrDefault(x => x.FieldName == "locations");
            var circleQuery = valueType.Strategy.MakeQuery(
                            new SpatialArgs(
                                SpatialOperation.Intersects,
                                valueType.Context.MakeCircle(
                                    double.Parse(searchModel.Longitude),
                                    double.Parse(searchModel.Latitude),
                                    DistanceUtils.Dist2Degrees(searchModel.RadiusInMiles, DistanceUtils.EarthMeanRadiusMiles))
                            )
                        );
            var shouldOrMust = searchModel.RestrictResultsToDistance ? Occur.MUST : Occur.SHOULD;
            geoQuery = new BooleanQuery();
            geoQuery.Add(circleQuery, shouldOrMust);

            if (geoQuery != null)
            {
                query.Query.Add(geoQuery, shouldOrMust);
            }
        }
        var result = searcher.Search(query.Query, searchModel.MaxResults, sort);
I am combining the spatial search with normal field search there
b
In the gist the an existing Lucene query is passed into
DoSpatialSearch()
method. and handle sorting by distance here: https://gist.github.com/bjarnef/6b427123903c90e034027bb8a3a5d43a#file-searchservice-cs-L67
d
Thanks all for all the examples! I'll try to read through them all and see if I can make it work
I see in your example that you used miles for the distance. Have you also tried it with kilometers by any chance? I noticed that my search somehow misinterprets miles and kilometers and the hacky fix was to use miles-to-kilometers conversion on my kilometer distance with
EarthMeanRadiusKilometers
for the radius.
c
i didn't try km, also i tested against online distance map tools and it was exactly the same results
2 Views