Failed to acquire write lock for id: -33...
h
Hi all! I've been looking into locking issues in Umbraco and I've found a possible cause. From what I can tell
ScopeProvider
is a singleton that manages all scopes (across all requests/threads) and seems to expect them to dispose in order. To top it off,
Scope
throws when it disposes if its "child" scope hasn't been disposed of, meaning any locks are not released 💥. Happy to hear everyone's thoughts, particularly if you think I'm wrong and/or know more about this area of Umbraco than me . https://harrygordon.co.uk/blog/not-thread-safe-obviously/ Here's the relevant thread on Github: https://github.com/umbraco/Umbraco-CMS/issues/14195#issuecomment-2117072274
@SavageNectar in terms of interim solutions, that's a good question. @Liam and I have discussed some options but we don't have anything concrete yet. I'll keep thinking about it over the weekend and see if we can come up with something. I should probably add a troubleshooting section to the article: the first thing I'd recommend is just check for any mishandling of
Scope
, I've seen it happen often enough in Umbraco projects. So anything like: - Uses of
Scope.CreateScope
outside of a
using
statement - Uses of scopes in parellel (not always bad but should be done with care, by the sounds of it) - Static variables for things that should be scoped (be it a scope or something that use a scope)
I am halfway through writing a static analyzer for calls to
CreateScope
outside of a using statement so I'll try finish that off next week.
s
Hi @Harry Gordon awesome, I have double-checked our code, and have no .CreateScope calls anywhere, I have an awesome issue now as well, where just moving an image to a new dir will result in a write lock time out
h
That sounds bonkers 😦. Are you seeing the child/ambient scope exception in the logs?
s
Umbraco.Cms.Core.DistributedLocking.Exceptions.DistributedWriteLockTimeoutException: Failed to acquire write lock for id: -334. at Umbraco.Cms.Persistence.SqlServer.Services.SqlServerDistributedLockingMechanism.SqlServerDistributedLock..ctor(SqlServerDistributedLockingMechanism parent, Int32 lockId, DistributedLockType lockType, TimeSpan timeout) at Umbraco.Cms.Persistence.SqlServer.Services.SqlServerDistributedLockingMechanism.WriteLock(Int32 lockId, Nullable`1 obtainLockTimeout) at Umbraco.Cms.Core.Scoping.LockingMechanism.ObtainWriteLock(Int32 lockId, Nullable`1 timeout) at Umbraco.Cms.Core.Scoping.LockingMechanism.LockInner(Guid instanceId, Dictionary`2& locks, HashSet`1& locksSet, Action`2 obtainLock, Nullable`1 timeout, Int32 lockId) at Umbraco.Cms.Core.Scoping.LockingMechanism.EagerWriteLockInner(Guid instanceId, Nullable`1 timeout, Int32[] lockIds) at Umbraco.Cms.Core.Scoping.LockingMechanism.WriteLock(Guid instanceId, Nullable`1 timeout, Int32[] lockIds) at Umbraco.Cms.Core.Scoping.CoreScope.WriteLock(Int32[] lockIds) at Umbraco.Cms.Core.Services.MediaService.Save(IMedia media, Int32 userId) at Umbraco.Cms.Web.BackOffice.Controllers.MediaController.PostAddFile(String path, String currentFolder, String contentTypeAlias, List`1 file)
that is the error I am getting when doing anything media related, looks like the first image moves or get added but anything additional fails, I do see that the first image lock stays on the db for quite a while, which makes me think the process is async and has not complete.
h
Hmmm, so I think this exception is just a symptom of the lock being unreleased. Are there any other exceptions or errors in the log from the time when you did the successful media operation? I'm assuming the media operation has got the lock and then somehow not released it.
As an aside, do you have any code that runs on media operations? Like
MediaSavedNotification
or something?
s
No other errors, I don't tie into the notifications for media, I use the Azure blob sdk which takes care of saving the raw file, and Umbraco only keeps the reference in the article to the image.
can different file format have an impact jpg vs webp or will all of them get treated the same?
h
Well what I'm wondering is if other
MediaService
operations result in the same thing... Or if it happens without Azure blob SDK
I am going to have a little dig around in
MediaService.Move
and see if I can find anything but it must be something specific to your project/environment/code though.
Is this happening in Azure? Are you sharing the DB with any other instances, including local?
s
So the solution is hosted on a azure sql vm, and only the sql db and the Umbraco instance is using the vm, we do have custom ingest api post for event, etc. I do see this happing, its almost as if the move or any media service will try to send and read all the images in a dir and sub dir's at once https://cdn.discordapp.com/attachments/1240964476254289930/1242145306422808717/image.png?ex=664cc576&is=664b73f6&hm=6fd525d4b302158c85550a462af948d82a75febb0971744fc12bc118b2c5b196&
h
I sent you a DM with some more thoughts/questions, no rush to reply though. I have a bit of time this week so I'm happy to help if I can.
753 Views