Yes, any red at the top is pretty terrible. Slightly less terrible if there is a big download saturating the link at the time ... but IMO the link should cope better than that. You really want no red up at the top. At all.
Perhaps I'm picky, but I'd prefer that there was almost no blue either (ie the average latency pretty much matches the minimum latency). Even better if this can be kept during large downloads.
Even pickier ... I'd prefer the green area to have an absolutely flat border. The minimum latency should always be identical.
Unfortunately, I think my pickiness disqualifies me from considering a VM connection.
My current BQM... doesn't quite live up to this pickiness. There's a hint of blue during saturation, and the occasional red dot.
The yellow spikes in there are (mostly) the places where my SamKnows testbox is running tests - some download tests saturating the link, and some testing streaming performance for YouTube and Netflix (it used to do iPlayer too).