The short answer is yes, you need to create your own nuget package from your modified Umbraco version.
The obvious follow up question is then; what modifications are you making that you can't make as a package? A package would make your life so much easier, eg. not having to keep your modified Umbraco repo up to date with new releases.