Boinc Credit - Cobbles, New, Screws, and Another New Idea?

Message boards : Number crunching : Boinc Credit - Cobbles, New, Screws, and Another New Idea?
Message board moderation

To post messages, you must log in.

1 · 2 · Next

AuthorMessage
Profile ML1
Volunteer moderator
Volunteer tester

Send message
Joined: 25 Nov 01
Posts: 13193
Credit: 7,508,002
RAC: 20
United Kingdom
Message 2048332 - Posted: 6 May 2020, 19:19:02 UTC
Last modified: 6 May 2020, 19:26:19 UTC

Dear All,

We have the present Boinc Credits 'rewards' scoring that has evolved from simply counting Work Units done (s@h Classic), to next award a score according to the assumed "Cobblestones" work done (Boinc), to finally our present "Credit New" scheme of inflationary Cobblestones scores.

There is always controversy surrounding this due to the consequences of how the scoring is awarded and how the Boinc system is 'gamed' for gaining a higher score. Both some of the Boinc users AND some of the Boing projects appear to game the scoring...

Is there a simple and compatible fix to keep everyone 'happy'?...


In brief, here's a quick thought that hopefully enjoys better consequences:

Proposal:

  • In the Boinc Manager that users install, balance the resource share between the attached projects according to the RAC for a host system.


(Part 2 is to redefine the scoring unit to better reflect the resource being used... That's for a follow-on post!)


This idea stems from my own observation for my spread of systems that despite setting equal shares, some projects score far higher than other projects. Hence, I'm guessing that the Boinc Manager proportions resource share according to CPU time allocated?... Instead, proportion according to the RAC so as to keep the scoring 'honest'...

A consequence of Credit New is that it acts to average out the awarded scores evenly across an 'average' of all types of hardware. I believe this has the effect of steadily continually reducing the score gained by older slower systems compared newer systems. Especially so as (high compute performance) GPUs gain ever greater prominence.

Also, are the higher scoring projects gaming the scoring to gain 'competitive users' and so causing an additional credits inflation?


Personal admission and bias:

I've always been unhappy with the definition of the Cobblestone that is used as a performance measure for the Boinc scoring. There is no measure there of how well whatever hardware is utilized... Worse still, equal weight is given to both integer and floating point operations, regardless of their relative real-world costs and individual performance...

Also note that the effect of running "Credit New" is to introduce an ever greater 'fiddle factor' that abstracts away from the initial Cobblestone scoring assumption...


OK, that's an initial thought.

Comments?

(Or has that idea already been thrashed to a death?...)


Keep searchin',
Martin


See new freedom: Mageia Linux
Take a look for yourself: Linux Format
The Future is what We all make IT (GPLv3)
ID: 2048332 · Report as offensive     Reply Quote
Grant (SSSF)
Volunteer tester

Send message
Joined: 19 Aug 99
Posts: 13179
Credit: 208,696,464
RAC: 304
Australia
Message 2048334 - Posted: 6 May 2020, 19:37:58 UTC - in response to Message 2048332.  
Last modified: 6 May 2020, 19:40:29 UTC

* In the Boinc Manager that users install, balance the resource share between the attached projects according to the RAC for a host system.
Won't work as each project uses it's own Credit mechanism, and even if all projects used Credit New the amounts of Credit will still vary due to work types, hardware & software etc.
Credit New does not work the way it was intended.



Hence, I'm guessing that the Boinc Manager proportions resource share according to CPU time allocated?
Nope, it uses REC., which is based on the Peak FLOPs for that project for that Host, which is probably based on the Peak FLOPs for it's Applications for that Host.



I've always been unhappy with the definition of the Cobblestone that is used as a performance measure for the Boinc scoring.
The problem isn't the definition of the Cobblestone, the problem is that the definition has been ignored, not only by individual projects but even by Credit New itself.
Grant
Darwin NT
ID: 2048334 · Report as offensive     Reply Quote
Sirius B Project Donor
Volunteer tester
Avatar

Send message
Joined: 26 Dec 00
Posts: 23578
Credit: 3,081,182
RAC: 7
Ireland
Message 2048337 - Posted: 6 May 2020, 20:05:14 UTC - in response to Message 2048332.  

(Or has that idea already been thrashed to a death?...)
Yes it has & also this would be better asked on the Boinc Dev board considering that this project will soon be in sleep mode.
ID: 2048337 · Report as offensive     Reply Quote
rob smith Crowdfunding Project Donor*Special Project $75 donorSpecial Project $250 donor
Volunteer moderator
Volunteer tester

Send message
Joined: 7 Mar 03
Posts: 20013
Credit: 416,307,556
RAC: 380
United Kingdom
Message 2048339 - Posted: 6 May 2020, 20:20:45 UTC

Thrashed to death, but like a Zombie it keeps coming back to life
Bob Smith
Member of Seti PIPPS (Pluto is a Planet Protest Society)
Somewhere in the (un)known Universe?
ID: 2048339 · Report as offensive     Reply Quote
Sirius B Project Donor
Volunteer tester
Avatar

Send message
Joined: 26 Dec 00
Posts: 23578
Credit: 3,081,182
RAC: 7
Ireland
Message 2048342 - Posted: 6 May 2020, 20:33:23 UTC - in response to Message 2048339.  

World War Z?
:-)

I believe I did make a suggestion In the past but I don't think it was even considered. Have a mechanism in place:
1 hr cpu/gpu run time = 10 to 50 credits
2 hr cpu/gpu run time 51 to 100 credits
,,,and so on.
The actual scale used would be down to project admins.
Both cpu & gpu crunchers benefit without any angst. CPU crunchers would have to accept that GPU crunchers crunch more so more credit.
For tasks completed under 1 hr, pro ratia them.
Job done.

Figures used are just examples before anyone jumps the gun.
ID: 2048342 · Report as offensive     Reply Quote
Profile ML1
Volunteer moderator
Volunteer tester

Send message
Joined: 25 Nov 01
Posts: 13193
Credit: 7,508,002
RAC: 20
United Kingdom
Message 2048343 - Posted: 6 May 2020, 20:35:36 UTC - in response to Message 2048334.  
Last modified: 6 May 2020, 20:36:40 UTC

Hence, I'm guessing that the Boinc Manager proportions resource share according to CPU time allocated?
Nope, it uses REC., which is based on the Peak FLOPs for that project for that Host, which is probably based on the Peak FLOPs for it's Applications for that Host...

Thanks for the link!

OK, so already done!...

So, the REC will take a few weeks to settle for a client. Hence a few random wobbles are to be expected when swapping between projects...



I've always been unhappy with the definition of the Cobblestone that is used as a performance measure for the Boinc scoring.

The problem isn't the definition of the Cobblestone, the problem is that the definition has been ignored, not only by individual projects but even by Credit New itself.

Strangely enough, the "Part 2" 'brainwave' is to reference the credit against physical reality...


Thanks again for the link.

Keep searchin',
Martin
See new freedom: Mageia Linux
Take a look for yourself: Linux Format
The Future is what We all make IT (GPLv3)
ID: 2048343 · Report as offensive     Reply Quote
Profile ML1
Volunteer moderator
Volunteer tester

Send message
Joined: 25 Nov 01
Posts: 13193
Credit: 7,508,002
RAC: 20
United Kingdom
Message 2048345 - Posted: 6 May 2020, 20:40:57 UTC - in response to Message 2048342.  

World War Z?
:-)

[...]

The actual scale used would be down to project admins.
Both cpu & gpu crunchers benefit without any angst. CPU crunchers would have to accept that GPU crunchers crunch more so more credit...

Credit Z?

Proportional upon the CPU cores / GPU compute elements used??...


Keep searchin',
Martin
See new freedom: Mageia Linux
Take a look for yourself: Linux Format
The Future is what We all make IT (GPLv3)
ID: 2048345 · Report as offensive     Reply Quote
Grant (SSSF)
Volunteer tester

Send message
Joined: 19 Aug 99
Posts: 13179
Credit: 208,696,464
RAC: 304
Australia
Message 2048346 - Posted: 6 May 2020, 20:46:46 UTC - in response to Message 2048342.  

World War Z?
:-)

I believe I did make a suggestion In the past but I don't think it was even considered. Have a mechanism in place:
1 hr cpu/gpu run time = 10 to 50 credits
2 hr cpu/gpu run time 51 to 100 credits
,,,and so on.
Or just pay Credit in accordance with the original definition of the Cobblestone.
Grant
Darwin NT
ID: 2048346 · Report as offensive     Reply Quote
Sirius B Project Donor
Volunteer tester
Avatar

Send message
Joined: 26 Dec 00
Posts: 23578
Credit: 3,081,182
RAC: 7
Ireland
Message 2048347 - Posted: 6 May 2020, 20:49:40 UTC - in response to Message 2048345.  

Credit Z?
Proportional upon the CPU cores / GPU compute elements used??...
Isn't that what is occurring atm?
I'm a CPU cruncher using 12 cores - 2x2 & 1x8. I'll lay odds that a newer faster CPU will crunch more than mine. Not only in the number of cores but speed as well. The same for GPU's unless I'm mistaken.
ID: 2048347 · Report as offensive     Reply Quote
Sirius B Project Donor
Volunteer tester
Avatar

Send message
Joined: 26 Dec 00
Posts: 23578
Credit: 3,081,182
RAC: 7
Ireland
Message 2048349 - Posted: 6 May 2020, 20:51:50 UTC - in response to Message 2048346.  

Or just pay Credit in accordance with the original definition of the Cobblestone.
Ah, the days of AP giving 800 to1000. :-)
ID: 2048349 · Report as offensive     Reply Quote
bluestar

Send message
Joined: 5 Sep 12
Posts: 4987
Credit: 2,084,789
RAC: 3
Message 2048351 - Posted: 6 May 2020, 21:04:53 UTC
Last modified: 6 May 2020, 21:06:24 UTC

My postings at PrimeGrid ends up in the drain, for not being readily visible, and that is not because of any credit either.

For one thing, that project could be almost dead, for only postings by other users, for still not any drain it could be for the same,
except that credit awarded with that project could be reflected or given in a similar way, for only the weighted it could be.

So perhaps rather amount instead, for just how much is given for a single task when only uploaded and reported, because a few tasks are making
for quite a bit of credit, for also that of run time length.

An exception here is PPS Sieve, which could be finished in maybe 10-30 minutes, depending on both CPU versus GPU, for also card for such, and some 3,371 back for each.

Only just top of the hat it could be, for also a couple of zombies, it could still be survival of the fittest, when only such a thing being offered.
ID: 2048351 · Report as offensive     Reply Quote
Profile Wiggo
Avatar

Send message
Joined: 24 Jan 00
Posts: 21699
Credit: 261,360,520
RAC: 489
Australia
Message 2048353 - Posted: 6 May 2020, 21:14:50 UTC - in response to Message 2048349.  

Or just pay Credit in accordance with the original definition of the Cobblestone.
Ah, the days of AP giving 800 to1000. :-)
Originally AP's paid 1344 cobblestones and doing MB work over the same time got you the same amount and back then Seti wasn't far off Einstein in awarded credit.

Cheers.
ID: 2048353 · Report as offensive     Reply Quote
Sirius B Project Donor
Volunteer tester
Avatar

Send message
Joined: 26 Dec 00
Posts: 23578
Credit: 3,081,182
RAC: 7
Ireland
Message 2048355 - Posted: 6 May 2020, 21:28:17 UTC - in response to Message 2048353.  
Last modified: 6 May 2020, 21:30:17 UTC

Never got many of those ones. In fact was very lucky to get any AP at all. :-(
Checking via Seti Spirit, the total AP tasks received in the life of this rig was 129 in 5 years.
Oops, got that wrong, 5 years.
ID: 2048355 · Report as offensive     Reply Quote
Profile ML1
Volunteer moderator
Volunteer tester

Send message
Joined: 25 Nov 01
Posts: 13193
Credit: 7,508,002
RAC: 20
United Kingdom
Message 2048356 - Posted: 6 May 2020, 21:33:51 UTC - in response to Message 2048351.  
Last modified: 6 May 2020, 21:34:12 UTC

My postings at PrimeGrid ends up in the drain, for not being readily visible, and that is not because of any credit either.

Stay focused. Stay on topic. Use short sentences?


... So perhaps rather amount instead, for just how much is given for a single task when only uploaded and reported, because a few tasks are making
for quite a bit of credit, for also that of run time length.

An exception here is PPS Sieve, which could be finished in maybe 10-30 minutes, depending on both CPU versus GPU, for also card for such, and some 3,371 back for each...

That is the problem. What is a good way to give credit? How do we add up the computer work done?

How do we measure what work has been done?!



Keep searchin',
Martin
See new freedom: Mageia Linux
Take a look for yourself: Linux Format
The Future is what We all make IT (GPLv3)
ID: 2048356 · Report as offensive     Reply Quote
Sirius B Project Donor
Volunteer tester
Avatar

Send message
Joined: 26 Dec 00
Posts: 23578
Credit: 3,081,182
RAC: 7
Ireland
Message 2048357 - Posted: 6 May 2020, 21:41:08 UTC - in response to Message 2048356.  

That is the problem. What is a good way to give credit? How do we add up the computer work done?
How do we measure what work has been done?!
Check out WCG.
Pretty good in my book.
Total run time
Points generated
Results returned
Badges.
ID: 2048357 · Report as offensive     Reply Quote
Grant (SSSF)
Volunteer tester

Send message
Joined: 19 Aug 99
Posts: 13179
Credit: 208,696,464
RAC: 304
Australia
Message 2048360 - Posted: 6 May 2020, 21:55:49 UTC - in response to Message 2048356.  

What is a good way to give credit?
According to the definition of the Cobblestone.



How do we add up the computer work done? How do we measure what work has been done?!
Since the cobblestone is based on FLOPS, FLOPS counting as per Seit before Credit new is best. Next best is an accurate estimate supplied by the project of the FLOPS necessary to process a given task.

As per Seti, some smoothing factor to allow for those Tasks that hare chewier or less tough to process than others.
Grant
Darwin NT
ID: 2048360 · Report as offensive     Reply Quote
Profile Keith Myers Special Project $250 donor
Volunteer tester
Avatar

Send message
Joined: 29 Apr 01
Posts: 12616
Credit: 1,160,866,277
RAC: 1,873
United States
Message 2048376 - Posted: 7 May 2020, 1:39:12 UTC - in response to Message 2048343.  

So, the REC will take a few weeks to settle for a client. Hence a few random wobbles are to be expected when swapping between projects...

Doesn't have to if you change the default setting of REC in the cc_config file which default is set for 10 days, hence the two week settling time commented upon.
Change the REC value to 1 day and the projects balance out much faster.
Seti@Home classic workunits:20,676 CPU time:74,226 hours

A proud member of the OFA (Old Farts Association)
ID: 2048376 · Report as offensive     Reply Quote
W-K 666 Project Donor
Volunteer tester

Send message
Joined: 18 May 99
Posts: 15717
Credit: 40,757,560
RAC: 67
United Kingdom
Message 2048385 - Posted: 7 May 2020, 4:10:47 UTC - in response to Message 2048346.  

World War Z?
:-)

I believe I did make a suggestion In the past but I don't think it was even considered. Have a mechanism in place:
1 hr cpu/gpu run time = 10 to 50 credits
2 hr cpu/gpu run time 51 to 100 credits
,,,and so on.
Or just pay Credit in accordance with the original definition of the Cobblestone.

If you think about it "Credit Screw" does just that.

The original Cobblestone definition was based around the RAC of an "average computer" doing "x" hours/day ("x" being fairly small, like one hour)
Today's "Credit Screw" continually adjusts based on the performance of today's "Average Computer".

As today's average computer is more powerful than yesterday's then the credit's awarded decrease, so that today the average computer gets the same RAC as what yesterdays average computer was awarded.

It's a pretty screwy system, but then again it matches my personal view of the person that thought it up.

Bring back "Flop Counting"
ID: 2048385 · Report as offensive     Reply Quote
rob smith Crowdfunding Project Donor*Special Project $75 donorSpecial Project $250 donor
Volunteer moderator
Volunteer tester

Send message
Joined: 7 Mar 03
Posts: 20013
Credit: 416,307,556
RAC: 380
United Kingdom
Message 2048415 - Posted: 7 May 2020, 16:34:03 UTC

If it matched the original definition of the "Cobblestone" as being based on the number of floating Point operations in a given time period by a theoretical processor then we wouldn't see the vast variation in credit awarded for a given run-time. What happens is the server takes a guess at the number of Floating Point Operations required to complete a work unit, then scales it by the apparent processing rate of the current host, and then applies a couple of fudge factors to try to compensate for the differences in processing time between the hypothetical and real times. But these take no notice of the fact that three things have happened over the years, first processors have become much faster, GPUs have become even faster still, and there have been some big steps in the way the calculations have actually been done, both at the silicone level (improved instruction sets) and the application (optimised applications for example); add to that a sort of feed-forward compensator that tries to predict what the processor will do next time around and we have the mess that is Credit New/Screw. In reality if the credit was awarded purely on the basis of the initial guess the situation would be much fairer, but one would have to take some action to cope with error-tasks (blank or noisy data). It is very noticeable that the majority of projects have implemented their own credit system, based either on a flat rate, or their own scheme as they recognise that any scheme reliant on a property of the host rather than the task is going to fail in some way or other.
Bob Smith
Member of Seti PIPPS (Pluto is a Planet Protest Society)
Somewhere in the (un)known Universe?
ID: 2048415 · Report as offensive     Reply Quote
Sirius B Project Donor
Volunteer tester
Avatar

Send message
Joined: 26 Dec 00
Posts: 23578
Credit: 3,081,182
RAC: 7
Ireland
Message 2048421 - Posted: 7 May 2020, 18:31:08 UTC

Seti Classic work units xx WU's = WCG results returned
Seti Classic CPU time xx hrs = WCG total run time

Start to finish the award structure for this project has been badly implemented.
2 things strike me with regards to WCG.
Points are allocated to each completed task.
IIRC, didn't WCG have its own software along the lines of F@H & moved to Boinc later?
Points are still generated but to cater for Boinc a factor of 7 is used.
Something which is accepted.
ID: 2048421 · Report as offensive     Reply Quote
1 · 2 · Next

Message boards : Number crunching : Boinc Credit - Cobbles, New, Screws, and Another New Idea?


 
©2021 University of California
 
SETI@home and Astropulse are funded by grants from the National Science Foundation, NASA, and donations from SETI@home volunteers. AstroPulse is funded in part by the NSF through grant AST-0307956.