Tuesday 6 November 2012

Idea For A Universal Way Of Processor Testing

This one is pretty nerdy but if widely accepted, it could end confusion once and for all.

Background:

I was just thinking about getting the nexus 4 today while driving. Then I thought, "how could I compare the Nexus 4 with the Galaxy Note II in a universal, quantifiable and uniform way?"

I haven't seen it done before, usually people just do a bunch of benchmarks and say that one phone "won" against another by the phone by "winning" more benchmarks, but the margin was never quantifiable. Anyway I came up with this method in the car lol, here it goes.

Method:

We run a set of 5 bench marks, all suited to testing different aspects of the processor (eg quadrant linpack etc).We then compare how well one phone performs with respect to the other and vice versa (we simply just divide one phone's score by the other's to achieve this).

This can be called the "R. Score" or Relative Score. The R. Score is naturally a percentage of how well one phone performs against another. 

For example, in Test A if we divide phone B's score phone C, and we get and R. Score of 1.25, this means that phone B performed 25% better than phone C for that test.


For the next step, we are only concerned with the phone with the highest Total R.Score. We subtract 1 from each R.score in each test and add them together and divide by 5 (basically just finding the average) This is now a measure of how much the winning phone outperforms the other in terms of percentage. Because this is done over 5 tests, it provides a fairly well-rounded overview of performance.

The full raw Scores are available here: http://www.mediafire.com/view/?32ri7n4by7ddvp2

Test score sources can be seen here:



http://androidandme.com/2012/11/smartphones-2/nexus-4-vs-htc-droid-dna-benchmark-showdown/attachment/smartbench-nexus4-droid-dna/

Advantages:
-If widely accepted, could provide an easy universal reference for phone comparisons.
-Unit Independent.
-Reduces sensitivity of tests that favour a certain type of processor.
-Anyone can carry this test out.

Disadvantages:
-It's quiet possible that no one actually cares.

4 comments:

  1. Hey James,

    I just happened upon your blog randomly tonight. After watching a few of your videos and scoping your profile, you seem to be an 'up and coming' vblogger or whatever the current term is. I just wanted to say, keep it up, I can smell the potential a mile away. A few more years of this tireless backbreaking work and I think you'll really make it.

    Sincerely,

    Nobody Important

    PS: Never lose the passion you have right now. Success is nothing without the spark that made it all possible.

    ReplyDelete
    Replies
    1. Thanks for the comment, it really meant a lot to me! I've been meaning to start writing more but I haven't had the time.

      I've actually been a bit torn lately. I have all these great ideas but i'm not sure if they'll fit with the origins of the channel. I'm actually thinking of breaking it into two separate channels..

      Anyway thanks again, and i'll always remember those last words.

      Delete
  2. In your video Samsung Galaxy Note 2 Replaces The PC you were using an app to monitor your processor, what is the name of that app?

    ReplyDelete
    Replies
    1. Hey Nathaniel,

      It's called "set cpu"

      Hope that helps!

      Delete