https://discord.umbraco.com logo
Join Discord
Powered by
Umbraco and Azure deployment slots
# help-with-umbraco
l
Hi all, We're running multiple Umbraco 13 sites, but some sites take quite some time to boot. We've optimized as much as possible (like UsePagedSql to false, production runtime mode etc), but the NuCache takes some time to load on these larger sites in combination with our custom logic. I know Umbraco 15 will probably be a lot faster, but we want to use deployment slots for web apps in Azure to minimize downtime. By itself, deployment slots are not hard to understand, but the issue I'm having is that I'm uncertain what to do with the database. We're using SQL Databases in Azure and they are not part of the deployment slot. Umbraco or Umbraco form updates run migrations on the database that could potentially break the old Umbraco version that running in the active deployment slot. So in that case there is still downtime, which defeats the point of deployment slots. Anyone has experience with this in combination with Umbraco? Or with deployment slots in Azure in combination with databases in general? Thanks in advance!
k
In general, the "old umbraco version" is running of it's cache and will be fine while you swap.
speaking of that cache though, that's where you might hurt yourself. In my distributed environment we've had to use the ignoreLocalDB for the nucache like so
Copy code
csharp
public class DisableNuCacheDatabaseComposer : IComposer
{
    public void Compose(IUmbracoBuilder builder)
    {
        var settings = new Umbraco.Cms.Infrastructure.PublishedCache.PublishedSnapshotServiceOptions
        {
            IgnoreLocalDb = true
        };
        builder.Services.AddSingleton(settings);
    }
}
As weird file-locking of the nucache unfortunately happen every so often in azure, either when we swap, or if we scale our frontend env. up
l
We don't have a distributed environment, so that will not be an issue 🙂 The issue is still that if the latest update fails for some reason and the updated Umbraco instance doesn't start, you're left with a database that in an incorrect state. I get what you are saying and I guess it's better than having no deployment slots, but still it doesn't feel right you know... Any idea if Umbraco 14+ (or 15 with it's hybrid cache) is working differently?
k
Interesting. We have weird nucache locks as well, and we don't even swap or scale. Not sure when it happens. What does 
IgnoreLocalDb
do exactly?
m
just be careful with the publisher.. as you can't have two publishers running both pointing at the same DB (which you might in a production slot and a deployment slot) and could be the root of you locking issues. One liner here to cover it..https://docs.umbraco.com/umbraco-cms/fundamentals/setup/server-setup/load-balancing/azure-web-apps#scaling
k
My understanding is it basically tells umbraco to forget about your nucache file, and store all of it in memory instead. It increases startup-time, but won't be able to file-lock. Memory is fuzzy but back when we were researching the issue, I believe what you might be experiencing is just Azure's normal app-service migration. In essense (if I'm remembering correct), Azure can at any given time basically decide "welp - this app has lived long enough on this server/environment, time to move it to another", and in doing so, you can end up with briefly having 2 app-services targeting the same nucache file (as your temp files aren't moved because they are in reality on a network-share and azure-magic just lets it seem to umbraco as if they are part of your file system). Azure moves your app (ie. it spins up a new one while it kills the old one) - the old one hasn't released it's lock yet on nucache from it's dying grasp - the new one, booting up, tries to access the nucache file but it's locked, and then kills itself.
Have not had the issue at all since running with ignorelocaldb, and bootup times aren't much of an issue due to our swap slot
l
Hmm, so it could help if we consider the 'old' and 'new' instance of Umbraco, which are running in the deployment slots as a load balanced instance? To prevent certain issues? I'm still not fond of having a single database that can break the current running instance. I thought I wasn't the first to do this and thought many other people would be running Umbraco using deployment slots lol 😛
k
We use slots but not for swapping. So we do have downtime during deployments, both ones that run migrations and others. But the downtime is so short we haven't even attempted to remove it using swapping. In your case, it sounds like * non-migrating deployments can be swapped against the same database, but * migrating deployments would cause a problem either way since the "Old Umbraco" slot would break when the database is upgraded by the "New Umbraco" slot.
k
@Luuk Peters (Proud Nerds) you are certainly not the first - we've run several sites with deployment slots, and also auto-scaling sites. The database just isn't the issue.
The content your frontend is serving is coming from the cache, not the database. If you have something running in background that could break your database and therefore your frontend, that sounds a bit like a architechture issue.
d
I love deployment slots for 0 downtime. I have asked HQ about similar functionality in cloud and apparently it's coming. In the meantime this is how I use deployment slots for Umbraco upgrades: https://wholething-limited.gitbook.io/umbraco/umbraco-upgrades/upgrade-an-umbraco-site
t
We did some initial testing of using slots when hosting on Azure. Seemed like a great idea. But we were swapping from a staging environment to prod environment, and the client has access to the staging env and use it to try out content stuff before publishing it, which does not work with slots very well. The reason being that during swapping you have a moment where the staging environment then points to the prod database and you might think you're editing stage, but you're in fact publishing in prod. We tried measuring the downtime when making a fully new deploy, and we couldn't even see it down from one version to the next despite hammering F5 in the browser 😄
l
In our case we just want to spin up the updated production version, while the current production version keeps running until the new production version is started correctly. We have some sites that take 5 minutes (or sometimes more) to boot. That's downtime that I think shouldn't happen these days with the possibilities we have. I'm warming to the idea that we still use one database that shouldn't have breaking changes, but still, uSync for instance could change content types that could break logic on the older site. However, worst case is that the old site is down for a moment, but that's downtime we would have anyway if we didn't use deployment slots. But in general, databases and deployment slots are still a risky thing, not specifically for Umbraco. Sure, with minor updates, the database should always be backwards compatible, nevertheless....risky. I would really love it if you can create a copy of the database automatically and swap that over as well.
t
I guess you could automate parts of that at least, creating a BACPAC and then importing that, but importing a BACPAC means you import it into a new database name, not just overwrite an existing one. MySQL is so much easier 😄
k
I mean - I suppose there is nothing stopping you from spinning up a copy of the database as part of your pipeline? I'm about 150 releases into a distributed setup with a frontend app-service with a swap and another appservice for the backoffice - but spinning up another DB on azure would just be "create database [my-very-important-database-2] as copy of [my-very-important-database-1]" or through azure CI - then as part of your pipeline you could just transform or appsettings to grab the correct database. When it hits swap - it will then run on 2, after swap, it will still run on 2 and you can delete 1 and repeat the process on every deploy.
l
But so far, you've never had database issues on deploy with a single database?
d
You can do all the database copying in Azure. In fact you can set all connection strings and configure them how you want e.g. swap with deploy or not. We often set up a sandbox db for testing content editing then point to another db. As you can tell I really like deployment slots.
k
No. More than 150 deploys so far on a single DB. Not even counting our staging env which also is both distributed and with swap slot as well to emulate production as close as possible
m
WEBSITE_DISABLE_OVERLAPPED_RECYCLING
set to 1, is a way to not have.. overlapping...
k
We still experienced it occasionally and you'll find plenty of frustrated people on various forums and threads having given up on configuring their way out of it and instead just plop the nucache in memory
But very happy if it works for you 🙂
m
As mentioned previously we don't use slots for the 
schedulingPublisher
too many pain points.. but for the 
subscribers
slot swapping seems ok.
b
I think for normal deployments we also didn't have many issues with swaps (only for the frontend app service though, not for 
schedulingPublisher
), however maybe for cases where you know you can need to have migrations/update wizards, you can do something more special and set up a pipeline to: - create a copy of the DB, - a slot for scheduling and subscriber app services all pointing to the copied DB - direct traffic to this new slot - do what needs to be done - if something goes wrong, have a way to just roll back - if all good, direct the traffic to the now working updated slot - clean up the DBs With the DB changes there would be a content editing freeze, but no downtime for customers
205 Views
PreviousNext
Powered by