My company is using Umbraco 8.7.0 on a single Elastic Beanstalk and we are looking to upgrade it to the load balanced configuration. We followed this guide (https://aws.amazon.com/blogs/devops/run-umbraco-cms-with-flexible-load-balancing-on-aws) to set up the beanstalks and now I'm to the point of needing to deploy it to this setup. I'm the Engineering Team Lead on the team, and am new to Umbraco so need a little help in terms of what the web.config values should be for the backoffice admin instance (single instance) and the load balanced instance. Do I need to change anything at all or just deploy the same application to both instances as is?

You need to configure the server roles for the publisher (single-instance admin) and subscribers (multi-instance frontend "readonly"). IIRC this works the same in 8 as in 10+, so the docs https://docs.umbraco.com/umbraco-cms/fundamentals/setup/server-setup/load-balancing/flexible-advanced#explicit-schedulingpublisher-server should apply. And IIRC

Web.config

changes aren't enough in 8 or 10+, you need a composer that sets the server role explicitly using code.