Skip versioning of content
# help-with-umbracor
Ramya Devendiran
05/15/2024, 9:04 AMWe import data programatically from external system (20000 records) into Umbraco node (includes Umbraco Commerce Price property). It takes 4 to 5 hrs to import data. During the import, the site is slow or even site crashes and shutdown. There is no need to maintain any of the content version on SAVEANDPUBLISH. Is there a way to prevent versioning content on a specific document type? I am looking for an improvement on the import background job.
Umbraco 12.
"ContentVersionCleanupPolicy": {
"EnableCleanup": true,
"KeepLatestVersionPerDayForDays": 1,
"KeepAllVersionsNewerThanDays": 1
}
c
Corné Hoskam
05/15/2024, 9:14 AMJust thinking out loud here, but if you're programmatically importing over 20k records into content nodes on a regular basis, perhaps it would be worth considering storing the imported data in database tables outside of Umbraco content? I don't know all of your use-cases of course, but at this scale manageability and performance could greatly improve if you were to target/manage said data outside of Umbraco Content, especially if there is currently a need to update that many content nodes on a regular basis with your current setup which has a lot of overhead
m
Mike Chambers
05/15/2024, 9:29 AMhttps://docs.umbraco.com/umbraco-cms/fundamentals/data/content-version-cleanup#overriding-global-settings if you have a specific doctype for your created imports?
r
Ramya Devendiran
05/15/2024, 9:33 AMThanks Corne and Mike for quick response. Import is not a scheduled job. It is triggered on demand. Say monthly once. We have also thought of alternatives to store outside of umbraco. But its tool late now to modify the impacts in webpages. Thanks Corne for your suggestion. @Mike, the link gives an option to prevent cleanup, I am looking for the other way round, need to maintain zero versions
m
Mike Chambers
05/15/2024, 9:34 AMyou can't maintain 0 version.. would always need 2.. current and published I think...
Mike Chambers
05/15/2024, 9:34 AMI think I see though you want finer granularity than a single day.../
Mike Chambers
05/15/2024, 9:34 AMperhaps you can replace https://github.com/umbraco/Umbraco-CMS/blob/contrib/src/Umbraco.Core/Services/DefaultContentVersionCleanupPolicy.cs with your own concrete implementation?
Mike Chambers
05/15/2024, 9:36 AMmaybe just changing over to hours rather than days.. but leave everything else as is.. remembering to change the globals as if hours đŸ™‚
if (age.TotalDays <=Mike Chambers
05/15/2024, 9:40 AMServices.AddUnique<IContentVersionCleanupPolicy, myNamespace.DefaultContentVersionCleanupPolicy>();Mike Chambers
05/15/2024, 9:41 AMOr via postpipeline composer?
Mike Chambers
05/15/2024, 9:42 AMSorry, I'm taking rubbish here this would still fire off the contentcleanup.. just removing at a faster rate of knots..
Mike Chambers
05/15/2024, 9:53 AMOn a different note.. presumable when you are importing content you are doing a lookup to update existing content.. are you using linq to find the matching content? maybe using examine could help??
Or are you reimporting your external content across the board?
I think as @Corné Hoskam has alluded too, if you are importing external content that isn't actually then required to be edited in the backoffice then you've hit an anti pattern?
https://docs.umbraco.com/umbraco-cms/reference/common-pitfalls#using-umbraco-content-items-for-volatile-data
One other suggestion there is the https://marketplace.umbraco.com/package/cmsimport. Has scheduled import options and handling ignore unchanged nodes.. (I know you have your own implementation but might be of interest)
Mike Chambers
05/15/2024, 10:04 AMJust thinking back about your initial ask
There is no need to maintain any of the content version on SAVEANDPUBLISHMike Chambers
05/15/2024, 10:07 AM(did I miss something though 5hours is 300mins.. is 18,000 seconds.. and you are getting a node created <1sec per node... . that seems pretty good!) does chunking the import help?
Mike Chambers
05/15/2024, 10:12 AMEven more complicated.. you could set up loadbalancing?? and control the distribution .. eg run the import against the schehuling publisher instance.. and reduce the distribution times to your subscriber..
https://docs.umbraco.com/umbraco-cms/reference/configuration/globalsettings#database-server-messenger
Mike Chambers
05/15/2024, 10:15 AMmight help.. not sure.... hope you find a solution đŸ™‚
r
Ramya Devendiran
05/15/2024, 12:23 PMThanks for your responses. I will look into your considerations and work it out
4 Views