How to deal with a bounding box that overlaps multiple Cloud-Optimized GeoTIFF tiles?

I'm working on a python script that takes a random bounding box as input, requests the relevant area of the latest intersecting Sentinel-2 COG, and creates a single NDVI raster with same the dimensions/coords as the input bounding box.

The problem I think I'm coming across is that the random bounding box sometimes intersects two different Sentinel COG tiles. I'm using Rasterio and creating a window for requesting the intersecting data for a given COG, but because the extent of the bounding box only partially intersects each COG, I'm getting errors.

Any high-level tips on how to approach this issue of trying read data from COGs when the desired bounding both overlaps two different COG tiles? I guess I could download the entirety of the two tiles, mosaic them, and then create a new NDVI raster from the desired bounding box, but that kind of defeats the purpose of COGs.

Thanks!