[Solved] How to search for a date
# help-with-umbraco
c
Umb 10.6.1 I have a search page which works fine for strings but I need to search for a date. The date comes as a string formatted yyyy-MM-dd. When I search for say 1965-05-02 I seem to get all records where the date has any of the numbers in it. I was just wondering how it's stored in the Examine Indexes. In the Settings section, if I do a search, I can see dates are shown as 02/05/1965 00:00:00 but I don't know if that's as a string or a DateTime object. Is it just a case of rearranging my incoming date string to match the Examine Index string or do I need to convert the incoming string to a DateTime object and search with that? Thanks.
h
It uses Ticks quick example
Copy code
csharp
            var today = DateTime.Now;
            long min = today.Ticks;
            long max = today.Ticks;
            query ??= "7d";

            switch (query)
            {
                case "30m" :
                    min = today.AddMinutes(-30).Ticks;
                    break;
                case "60m" :
                    min = today.AddHours(-1).Ticks;
                    break;
                case "1d" :
                    min = today.AddDays(-1).Ticks;
                    break;
                case "7d" :
                    min = today.AddDays(-7).Ticks;
                    break;
                case "1m" :
                    min = today.AddMonths(-1).Ticks;
                    break;
                case "1y" :
                    min = today.AddYears(-1).Ticks;
                    break;
            }

            int pageIndex = page - 1;
            if(pageIndex < 0) {pageIndex = 0;}
            int pageSize = CurrentPage.Value<int>("intPageSize");

            if (_examineManager.TryGetIndex("ForumIndex", out var index))
            {
                
                var searcher = index.Searcher;

                var examineQuery = searcher.CreateQuery(IndexTypes.Content)
                .Field("postType", "Topic")
                    .And().RangeQuery<long>(new string[] { "updated" }, min, max)
                    .OrderByDescending(new SortableField[] { new SortableField("updateDate") });
                
                results = examineQuery.Execute();
            }
c
Ah right. So I'll have to convert my Date string to a DateTime object to get the ticks and compare on that. Thanks 🙂 That explains why I currently get what I get, lol
Hmmm, tried that, gives me an error
Copy code
System.InvalidOperationException: Could not perform a range query on the field deceasedDateOfBirth, it's value type is Examine.Lucene.Indexing.FullTextType
   at Examine.Lucene.Search.LuceneSearchQuery.<>c__DisplayClass13_0`1.<RangeQueryInternal>b__0()
The code is
Copy code
searchQuery.And().Field("deceasedDateOfBirth", searchDOBTicks);
where "searchDOBTicks" can be ticks or string as I try out different options....... I'm not trying to do a range query I just want a match with what looks like "02/05/1965 00:00:00" in the Index. I've also tried converting my "1965-05-02" to both "02/05/1965" and "02/05/1965 00:00:00" as well as Ticks but nothing works so far 😦
The date is orginally from a DatePicker, which the docs say "The value saved is a standard DateTime value but does not contain time information.", so no idea where the time part is coming from. But is it saved as a string? The index seems to be telling me that.
In the Examine Manager, External Index Search, +deceasedDateOfBirth:"05/06/1934" gives the right, single, answer. But putting "05/06/1934" in the code returns 5 answers, only one is correct. 🤔
d
I think the field query doesn't add the double quotes around your search string. If I remember correctly, you'd have to do something like
"05/06/1934".Explicit()
to actually search for the whole string, rather than the individual components.
c
The date can only be part of the query so don't want the whole thing to be "Explicit". I don't get an "Explicit" available anyway, either on the Add or the Execute.
n
I'd be looking v closely at Callum's Umbraco search extensions package, which will smooth over these sorts of Examine issues. From memory, it will help with dates by indexing as easily searchable values, to avoid gymnastics in your queries.
c
As there's only up to 3 parameters I was trying to do it as a lamba query for children but couldn't get that syntax right either. I thought Examine might be better, lol
h
I think that is why I ended up using ticks as couldn't get the filter to work, mine is a custom index though, so will check what I did when I'm back home (I may me adding to the index as long not date)
c
Reverted to foreach loops around the collection of pages with the dates in, which worked for individual dates, but not good for combinations, i.e. any combination of Name1, Name2, Date1, Date2. So still struggling 😦 If only Examine/Lucine respected dates.
Back to Examine again (on advice, not a whim I hasten to add!). Suggested to use a RangeQuery to look for a date >= MyDate AND date <= MyDate in order to hit just "MyDate". Best I can think of doing is :-
Copy code
if (!searchDOB.IsNullOrWhiteSpace()) { 
            DateTime searchDOBDT = new DateTime(Int32.Parse(searchDOB.Split("-")[0]), Int32.Parse(searchDOB.Split("-")[1]), Int32.Parse(searchDOB.Split("-")[2]));
            searchQuery.And().RangeQuery<DateTime>(new[] {"deceasedDateOfBirth"}, searchDOBDT, searchDOBDT); 
        }
Which gives the expected error:
InvalidOperationException: Could not perform a range query on the field deceasedDateOfBirth, it's value type is Examine.Lucene.Indexing.FullTextType
. So I guess I'll have to workout how to make the date picker save as a DateTime instead of a String, if that's even possible.
n
Index the date as ticks (search extensions package will do this for you) and this is straightforward.
c
Installed https://github.com/callumbwhyte/umbraco-search-extensions and rebuilt indexes. Still says the DOB field (which is from a date picker) is FullTextType 😦 The package only states that createDate and updateDate are converted to Date values. I suspect I'll have to do some composing to get the DOB field into the index as a separate field and as a Date, so not that straightforward, unless I've missed something?
So I implemented this (and registered it in startup.cs), rebuilt indexes and it still reports the DOB field as FullTextType:-
Copy code
public class SearchIndexOptions : IConfigureNamedOptions<LuceneDirectoryIndexOptions>
{
    public void Configure(string name, LuceneDirectoryIndexOptions options)
    {
        if (name == "ExternalIndex")
        {
            options.FieldDefinitions.AddOrUpdate(new FieldDefinition("deceasedDateOfBirth", "Date"));
        }
    }

    public void Configure(LuceneDirectoryIndexOptions options) {
        throw new NotImplementedException();
    }
}
And changing the searcher to
searchQuery.And().RangeQuery<long>(new[] {"deceasedDateOfBirth"}, searchDOBDT.Ticks, searchDOBDT.Ticks);
doesn't help either.
Checking with Luke, the createDate and updateDate both appear to have values like the left image. While the DOB field, which as far as I'm concerned should now look the same, looks like the right image. Even though I've deleted the ExamineIndexes folder and restarted and rebuilt indexes. It's as if the code above isn't doing anything, though it's the same, or similar, code to what the package uses to convert the createDate and updateDate fields. I must be missing something simple. It's registered with services.AddTransient();
m
think it's your field definition that's wrong.. the magicstring is
"datetime"
though I'd use..
options.FieldDefinitions.AddOrUpdate(new FieldDefinition("deceasedDateOfBirth", FieldDefinitionTypes.DateTime));
from the examine namespace to avoid
and maybe also
var deceasedDateOfBirth = {DocType}.GetModelPropertyType(_publishedSnapshotAccessor, x => x.deceasedDateOfBirth)!.Alias;
if you are using models builder?
makes things more robust, or at least an early warning if changes are made 😉
c
I was just following the example given in the package repo, but I'll give it a go, thanks.
m
iteresting that the repo uses "date" I wonder if it's beacuse createDate is already a "datetime" in the core, so that's just ignored, as an unknown fieldtype? https://github.com/umbraco/Umbraco-CMS/blob/contrib/src/Umbraco.Infrastructure/Examine/UmbracoFieldDefinitionCollection.cs#L22
c
Doesn't make any difference. I also can't hit the breakpoint indicated below. Which might indicate why it's not working but I'm not sure when this would kick in. If it's on site startup then I'm not sure it's possible to switch into debug so soon anyway.
Copy code
public class ConfigureIndexOptions : IConfigureNamedOptions<LuceneDirectoryIndexOptions>
{
    private readonly IPublishedSnapshotAccessor _publishedSnapshotAccessor;

    public ConfigureIndexOptions(IPublishedSnapshotAccessor publishedSnapshotAccessor) {
        _publishedSnapshotAccessor = publishedSnapshotAccessor;
    }
    
    public void Configure(string name, LuceneDirectoryIndexOptions options)
    {
        if (name == "ExternalIndex")  //Breakpoint here not hit
        {
            var deceasedDateOfBirth = MemorialPage.GetModelPropertyType(_publishedSnapshotAccessor, x => x.DeceasedDateOfBirth)!.Alias;
            var deceasedDateOfDeath = MemorialPage.GetModelPropertyType(_publishedSnapshotAccessor, x => x.DeceasedDateOfDeath)!.Alias;
            options.FieldDefinitions.AddOrUpdate(new FieldDefinition(deceasedDateOfBirth, FieldDefinitionTypes.DateTime));
            options.FieldDefinitions.AddOrUpdate(new FieldDefinition(deceasedDateOfDeath, FieldDefinitionTypes.DateTime));
        }
    }

    public void Configure(LuceneDirectoryIndexOptions options) {
        throw new NotImplementedException();
    }
}
m
are you composing, or injecting in startup.cs?
public class IndexOptionsComposer : IComposer { public void Compose(IUmbracoBuilder builder) { // Custom Examine configuration builder.Services.ConfigureOptions(); } }
c
services.AddTransient(); In startup.cs
I could add a composing class on the bottom if it'd make a difference.
m
not sure.. but you aren't adding a service..
c
?
m
services.AddTransient<ConfigureIndexOptions>();
which this is doing.
c
I'll try the composer then.
c
Ooooh, now I get !Unhandled exception.
Copy code
System.InvalidOperationException: Wasn't possible to a get a valid Snapshot
   at Umbraco.Extensions.PublishedSnapshotAccessorExtensions.GetRequiredPublishedSnapshot(IPublishedSnapshotAccessor publishedSnapshotAccessor)
on start up
I'll take that out, revert to magic strings and try again
m
if you did want it in startup.cs,, think you need to extend the umbracoBuilder like this.. https://docs.umbraco.com/umbraco-cms/reference/routing/request-pipeline/icontentfinder#adding-and-removing-icontentfinders
c
I'm not fussed, I just want it to work, lol
m
For me I find using composers where ever possible (and in a separate code library) means when I update umbraco I can just leave startup.cs alone and compare to the new version startup.cs (as in place upgrade never touches startup.cs).. also stops the crap forgot to add in startup.cs! 😉
c
It works!!!!!!!!
The index suddenly changed to look like the one on the left with odd chars rather than the numerals it had before and now it returns proper results on a date search!
So it was the composing thing that was out. I just thought registration was registration, but apparently not. I'll have to read up on the difference in those links you gave me. The final code is ...
Copy code
public class ConfigureIndexOptions : IConfigureNamedOptions<LuceneDirectoryIndexOptions>
{
    public void Configure(string name, LuceneDirectoryIndexOptions options)
    {
        if (name == "ExternalIndex")
        {
            options.FieldDefinitions.AddOrUpdate(new FieldDefinition("deceasedDateOfBirth", FieldDefinitionTypes.DateTime));
            options.FieldDefinitions.AddOrUpdate(new FieldDefinition("deceasedDateOfDeath", FieldDefinitionTypes.DateTime));
        }
    }

    public void Configure(LuceneDirectoryIndexOptions options) {
        throw new NotImplementedException();
    }
}

public class IndexOptionsComposer : IComposer
{
    public void Compose(IUmbracoBuilder builder)
    {
        // Custom Examine configuration
        builder.Services.ConfigureOptions<ConfigureIndexOptions>();
    }
}
Thanks very much. Are you going to UK Fest?
m
one more for you..
Constants.UmbracoIndexes.ExternalIndexName
c
I always think, what's the point, becuase it's always going to be "ExternalIndex"
I can't think of a scenario when it would change
m
till we get index abstraction.. and it becomes
ExternalLuceneIndex
vs
ExternalElaticIndex
😉
c
I see your point, but we're not there yet. And I'm not of that level of ability to go swapping out search providers, lol
m
but yeah.. those it's always been that it's never gonna change.. even the core has "nodeName" as a magic string everywhere 🙂
38 Views