Hi, is there any way to prevent/remove a field from in the ExternalIndex

Yep, Shannon blogged about it: https://shazwazza.com/post/filtering-fields-dynamically-with-examine/

Could you also configure using the

ContentValueSetValidator

, has include/exclude field support now?

The IValueSetValidator is also responsible for filtering the data in the ValueSet. For example, by default the validator for the MemberIndex will validate on all the default member properties, so an extra property "PhoneNumber", would not pass validation, and therefore not be included.

https://github.com/umbraco/Umbraco-CMS/blob/contrib/src/Umbraco.Infrastructure/Examine/ContentValueSetValidator.cs

csharp
 public ContentValueSetValidator(
        bool publishedValuesOnly,
        bool supportProtectedContent,
        IPublicAccessService? publicAccessService,
        IScopeProvider? scopeProvider,
        int? parentId = null,
        IEnumerable<string>? includeItemTypes = null,
        IEnumerable<string>? excludeItemTypes = null,
        IEnumerable<string>? includeFields = null,
        IEnumerable<string>? excludeFields = null)
        : base(includeItemTypes, excludeItemTypes, includeFields, excludeFields)
    {

So should be able to use the

csharp
 IConfigureNamedOptions<LuceneDirectoryIndexOptions>
{
public void Configure(string? name, LuceneDirectoryIndexOptions options)
{
    switch (name)
    {
        //NB you need to rebuild the examine index for these changes to take effect
        case Constants.UmbracoIndexes.ExternalIndexName:
            options.Validator = new ContentValueSetValidator(true, true, _publicAccessService, _scopeProvider, {included fields}, {excluded fields});
            break;
    }
}

}

????

Probably, I did it during a TransformingIndexEvent where I got the field collection, removed some fields and overwrote it with the reduced version, but Shannon commented and linked his blogpost so I figured that is the "official" way

Would be easier if the fieldDefinitions wasn't a readonly collection, and could simply do

FieldDefintions.TryRemove()

though though prob a reason why you can add/alter to the readonly collection.. but not remove. 😉

Yes FiedlDefinitions doesnt help

I also have a memory of readng about being able to turn off the blockgrid extra fields being generated.. maybe that has some pointers?

yes, I have that

that's on the https://github.com/umbraco/Umbraco-CMS/blob/contrib/src/Umbraco.Core/PropertyEditors/JsonPropertyIndexValueFactoryBase.cs I think.. so depending on if all your unwanted index values are a particular datatype might be an option too?

Yea what I did initially is this in a TransformingIndexValues event:

csharp
var rawValuesDictionary = e.ValueSet.Values?.ToDictionary(
    x => x.Key, 
    x => x.Value.ToList());

var valuesDictionary = rawValuesDictionary?.Where(
    x => !x.Key.Contains("items[")).ToDictionary(
        x => x.Key, x => x.Value);

e.SetValues(valuesDictionary?.ToDictionary(
    x => x.Key, 
    x => (IEnumerable<object>)x.Value));

So I just use linq to go through fields containing a string and remove them

weirdly, I cannot see that constructor with excludefields

is it version related?

I am using 10.7

Looks like 12.1 and up.

Yes thats why I am confused

on the indexfactory.. you can do something like..

csharp
using Umbraco.Cms.Core.Composing;
using Umbraco.Cms.Core.PropertyEditors;

namespace Umbraco.Cms.Web.UI.Custom;

public class MyBlockValuePropertyIndexValueFactory : DefaultPropertyIndexValueFactory, IBlockValuePropertyIndexValueFactory
{
}

public class MyBlockValuePropertyIndexValueFactoryComposer : IComposer
{
    public void Compose(IUmbracoBuilder builder) => builder.Services.AddSingleton<IBlockValuePropertyIndexValueFactory, MyBlockValuePropertyIndexValueFactory>();
}

I think this would allow you to alter how a block value would write into the index? though maybe not remove the actual property?

I am now thinking whether i should completley get rid of that particular doctype