witholding samples is not an issue if you buy your parts anonymously at retail which is what reviewers should be doing.
If you buy what you reviewing at retail, then you got no obligations to follow any guide of "review guide", you get retail parts the same as consumers, not engineering samples, or cherry picked of the production line.
Buying retail also you may get late enough to get a part that has had bait and switch of components happen, which if it affects consumers it should be reflected in reviews.
There has been a ton of problems I observed lately in reviews, as you say this practice has gone on for a while but its just something I have picked on more lately.
1 - Most Vega reviews witholding 1080ti from results, even tho vega 64 cards were in 1080ti price range, I argued with kyle on hardforum about this, his reasoning was that the retail price was been price gouged and that isnt AMD's fault. I said reviews should reflect reality for consumers, and his response was in lala land, we never came to agreement. Gibbo (ocuk staff member in control of stock) also responded to some of these claims saying if he sold at the prices hardocp claimed were what should be sold at then his company would be selling at a loss, suggesting this price gouging is not just at retail level.
2 - Overclocked intel CPU's witheld from most ryzen 2 test results, in addition features like MCE disabled for intel systems but equivelent features like PE enabled for ryzen 2.
3 - On the other side, reviewers claiming they testing intel at stock but actually testing with MCE enabled, giving inflated results.
4 - Also on intel, reviewers testing 8400 intel CPUs on high end ROG boards with TDP limits removed on a non overclocking chip.
5 - GN deciding to test gaming and streaming performance on a ryzen 2600x and compare it to the 8600k., this one really got my interest, mainly because its the first time I seen such a thing tested by anyone, secondly he was testing using uncapped game framerate and software based encoding, which is a highly unusual config that game streamers would use, I concluded GN did this test because he knew it would put the 2600X in the best possible light vs the 8600k, its the ultimate perfect workload for it. The problem I have with this is the reviewer went out his way to find a test to put the product he is testing in the best light possible, meaning that the reviewer feels he has a job to do that and hence bias. There is absolutely no doubt GN will not do these software based streaming tests when he next reviews an intel i5 chip, if he did he may get put on intels blacklist.
6 - When intel gets reviewed the intel OC performance results will be on the graph to help out intel, when AMD gets reviewed, they will be omitted to help out AMD.
7 - when the Nvidia maxwell cards got reviewed, people observed there was a very limited set of other products they were put alongside on the graphs, it was as if nvidia only wanted that visual comparison to be made, turns out people were right, pcper confirmed it was at nvidia's request.
8 - Certain products never get reviewed, simply because the company decides not to send review samples out of those products, ok this is not strictly true, smaller rviewers who actually have to pay for what they review, might buy these products, but generally they get skipped over by the well known reviewers, an example is low end motherboards, especially from asus, 5 years ago the mainstream range of asus boards used to get reviewed, but now they dont, perhaps its not surprising that feature wise they have been significantly downgraded and its not in asus's interest for these boards to get reviewed. OC3D recently discovered the asus z370 strix boards had broken LLC in their bios, he reported it, they ignored it, he waited, nothing happened, he published a video saying he cannot review the boards because they broken and all hell broke loose. Asus eventually fixed the bios, but it turns out all the asus prime boards had the same bug, however they did not get a bios fix for this reason stated by raja @ asus.
"The boards dont have the hardware spec for the feature to work" So basically asus sold overclocking boards that cannot utilise standard overclocking features because they were under spec'd.
9 - The games tested on reviews are games that are optimised for high core cpu's, the problem is only 8% of games (that have been analysed) on steam can use more than 2 cpu cores at once, yet 96% of games reviewed in the past 12 months can. This is clearly to push a message that cpu's with high thread counts are great for gaming, intel and AMD both want this pushed so it gets done, e.g. intel want people to buy i7's instead of i5's and AMD wants people to buy their thread heavy cpu's as well. However even with this bias on the reviews, 8600k's keep up with 8700k's and the 2600x keeps up with the 2700x.
There is more but I feel I have made my point.
Adored did an excellent video analysing the chances of getting a "golden sample" 8700k at retail based on the silicon lottery website statistics, silicon lottery is a website that tests loads of cpus, rates them for a specific speed, and sells tham for that speed. It gives "golden sample" something like a 3% chance of finding one. Adored analysed the cpu's reviewed by a bunch of reviewers and found more than half had "golden samples".
The reviewer in the past 12 months I have found with the most integrity is bizarrely OC3d. He talks to his viewers like they kindergarden But he is the only reviewer that did all of the following.
1 - Reported reviewers were using MCE on stock intel tests, and that he even tried to get the default changed to disabled (unsuccessfully). On his intel cpu graphs he has 3 results, stock, MCE and OC, so all 3 scenarios covered, including on AMD reviews.
2 - He had the 1080ti on his vega review graphs, he didnt talk about it, but it was at least on the graph.
3 - He came out with the broken strix motherboard video, and also done another recently for a gigabyte board.
4 - He said clock speed is king in games on his AMD review, I dont think AMD would have liked that very much, but its an accurate statement.
Edited by Chrysalis (Tue 24-Apr-18 21:00:07)