Dear All,

We have the present Boinc Credits 'rewards' scoring that has evolved from simply counting Work Units done (s@h Classic), to next award a score according to the assumed "Cobblestones" work done (Boinc), to finally our present "Credit New" scheme of inflationary Cobblestones scores.

There is always controversy surrounding this due to the consequences of how the scoring is awarded and how the Boinc system is 'gamed' for gaining a higher score. Both some of the Boinc users AND some of the Boing projects appear to game the scoring...

Is there a simple and compatible fix to keep everyone 'happy'?...


In brief, here's a quick thought that hopefully enjoys better consequences:

Proposal:

• In the Boinc Manager that users install, balance the resource share between the attached projects according to the RAC for a host system.
  

(Part 2 is to redefine the scoring unit to better reflect the resource being used... That's for a follow-on post!)


This idea stems from my own observation for my spread of systems that despite setting equal shares, some projects score far higher than other projects. Hence, I'm guessing that the Boinc Manager proportions resource share according to CPU time allocated?... Instead, proportion according to the RAC so as to keep the scoring 'honest'...

A consequence of Credit New is that it acts to average out the awarded scores evenly across an 'average' of all types of hardware. I believe this has the effect of steadily continually reducing the score gained by older slower systems compared newer systems. Especially so as (high compute performance) GPUs gain ever greater prominence.

Also, are the higher scoring projects gaming the scoring to gain 'competitive users' and so causing an additional credits inflation?


Personal admission and bias:

I've always been unhappy with the definition of the Cobblestone that is used as a performance measure for the Boinc scoring. There is no measure there of how well whatever hardware is utilized... Worse still, equal weight is given to both integer and floating point operations, regardless of their relative real-world costs and individual performance...

Also note that the effect of running "Credit New" is to introduce an ever greater 'fiddle factor' that abstracts away from the initial Cobblestone scoring assumption...


OK, that's an initial thought.

Comments?

(Or has that idea already been thrashed to a death?...)


Keep searchin',
Martin

---

See new freedom: Mageia Linux
Take a look for yourself: Linux Format
The Future is what We all make IT (GPLv3)

* In the Boinc Manager that users install, balance the resource share between the attached projects according to the RAC for a host system.

Won't work as each project uses it's own Credit mechanism, and even if all projects used Credit New the amounts of Credit will still vary due to work types, hardware & software etc.
Credit New does not work the way it was intended.

Hence, I'm guessing that the Boinc Manager proportions resource share according to CPU time allocated?

Nope, it uses REC., which is based on the Peak FLOPs for that project for that Host, which is probably based on the Peak FLOPs for it's Applications for that Host.

I've always been unhappy with the definition of the Cobblestone that is used as a performance measure for the Boinc scoring.

The problem isn't the definition of the Cobblestone, the problem is that the definition has been ignored, not only by individual projects but even by Credit New itself.

---

Grant
Darwin NT

(Or has that idea already been thrashed to a death?...)

Yes it has & also this would be better asked on the Boinc Dev board considering that this project will soon be in sleep mode.

Thrashed to death, but like a Zombie it keeps coming back to life

---

Bob Smith
Member of Seti PIPPS (Pluto is a Planet Protest Society)
Somewhere in the (un)known Universe?

World War Z?
:-)

I believe I did make a suggestion In the past but I don't think it was even considered. Have a mechanism in place:
1 hr cpu/gpu run time = 10 to 50 credits
2 hr cpu/gpu run time 51 to 100 credits
,,,and so on.
The actual scale used would be down to project admins.
Both cpu & gpu crunchers benefit without any angst. CPU crunchers would have to accept that GPU crunchers crunch more so more credit.
For tasks completed under 1 hr, pro ratia them.
Job done.

Figures used are just examples before anyone jumps the gun.

Hence, I'm guessing that the Boinc Manager proportions resource share according to CPU time allocated?

Nope, it uses REC., which is based on the Peak FLOPs for that project for that Host, which is probably based on the Peak FLOPs for it's Applications for that Host...

Thanks for the link!

OK, so already done!...

So, the REC will take a few weeks to settle for a client. Hence a few random wobbles are to be expected when swapping between projects...

I've always been unhappy with the definition of the Cobblestone that is used as a performance measure for the Boinc scoring.

The problem isn't the definition of the Cobblestone, the problem is that the definition has been ignored, not only by individual projects but even by Credit New itself.

Strangely enough, the "Part 2" 'brainwave' is to reference the credit against physical reality...


Thanks again for the link.

Keep searchin',
Martin

---

See new freedom: Mageia Linux
Take a look for yourself: Linux Format
The Future is what We all make IT (GPLv3)

World War Z?
:-)

[...]

The actual scale used would be down to project admins.
Both cpu & gpu crunchers benefit without any angst. CPU crunchers would have to accept that GPU crunchers crunch more so more credit...

Credit Z?

Proportional upon the CPU cores / GPU compute elements used??...


Keep searchin',
Martin

---

See new freedom: Mageia Linux
Take a look for yourself: Linux Format
The Future is what We all make IT (GPLv3)

World War Z?
:-)

I believe I did make a suggestion In the past but I don't think it was even considered. Have a mechanism in place:
1 hr cpu/gpu run time = 10 to 50 credits
2 hr cpu/gpu run time 51 to 100 credits
,,,and so on.

Or just pay Credit in accordance with the original definition of the Cobblestone.

---

Grant
Darwin NT

Credit Z?
Proportional upon the CPU cores / GPU compute elements used??...

Isn't that what is occurring atm?
I'm a CPU cruncher using 12 cores - 2x2 & 1x8. I'll lay odds that a newer faster CPU will crunch more than mine. Not only in the number of cores but speed as well. The same for GPU's unless I'm mistaken.

Or just pay Credit in accordance with the original definition of the Cobblestone.

Ah, the days of AP giving 800 to1000. :-)

My postings at PrimeGrid ends up in the drain, for not being readily visible, and that is not because of any credit either.

For one thing, that project could be almost dead, for only postings by other users, for still not any drain it could be for the same,
except that credit awarded with that project could be reflected or given in a similar way, for only the weighted it could be.

So perhaps rather amount instead, for just how much is given for a single task when only uploaded and reported, because a few tasks are making
for quite a bit of credit, for also that of run time length.

An exception here is PPS Sieve, which could be finished in maybe 10-30 minutes, depending on both CPU versus GPU, for also card for such, and some 3,371 back for each.

Only just top of the hat it could be, for also a couple of zombies, it could still be survival of the fittest, when only such a thing being offered.

Or just pay Credit in accordance with the original definition of the Cobblestone.

Ah, the days of AP giving 800 to1000. :-)

Originally AP's paid 1344 cobblestones and doing MB work over the same time got you the same amount and back then Seti wasn't far off Einstein in awarded credit.

Cheers.

Never got many of those ones. In fact was very lucky to get any AP at all. :-(
Checking via Seti Spirit, the total AP tasks received in the life of this rig was 129 in 5 years.
Oops, got that wrong, 5 years.

My postings at PrimeGrid ends up in the drain, for not being readily visible, and that is not because of any credit either.

Stay focused. Stay on topic. Use short sentences?

... So perhaps rather amount instead, for just how much is given for a single task when only uploaded and reported, because a few tasks are making
for quite a bit of credit, for also that of run time length.

An exception here is PPS Sieve, which could be finished in maybe 10-30 minutes, depending on both CPU versus GPU, for also card for such, and some 3,371 back for each...

That is the problem. What is a good way to give credit? How do we add up the computer work done?

How do we measure what work has been done?!



Keep searchin',
Martin

---

See new freedom: Mageia Linux
Take a look for yourself: Linux Format
The Future is what We all make IT (GPLv3)

That is the problem. What is a good way to give credit? How do we add up the computer work done?
How do we measure what work has been done?!

Check out WCG.
Pretty good in my book.
Total run time
Points generated
Results returned
Badges.

What is a good way to give credit?

According to the definition of the Cobblestone.

How do we add up the computer work done? How do we measure what work has been done?!

Since the cobblestone is based on FLOPS, FLOPS counting as per Seit before Credit new is best. Next best is an accurate estimate supplied by the project of the FLOPS necessary to process a given task.

As per Seti, some smoothing factor to allow for those Tasks that hare chewier or less tough to process than others.

---

Grant
Darwin NT

So, the REC will take a few weeks to settle for a client. Hence a few random wobbles are to be expected when swapping between projects...

Doesn't have to if you change the default setting of REC in the cc_config file which default is set for 10 days, hence the two week settling time commented upon.
Change the REC value to 1 day and the projects balance out much faster.

---

Seti@Home classic workunits:20,676 CPU time:74,226 hours

A proud member of the OFA (Old Farts Association)

World War Z?
:-)

I believe I did make a suggestion In the past but I don't think it was even considered. Have a mechanism in place:
1 hr cpu/gpu run time = 10 to 50 credits
2 hr cpu/gpu run time 51 to 100 credits
,,,and so on.

Or just pay Credit in accordance with the original definition of the Cobblestone.

If you think about it "Credit Screw" does just that.

The original Cobblestone definition was based around the RAC of an "average computer" doing "x" hours/day ("x" being fairly small, like one hour)
Today's "Credit Screw" continually adjusts based on the performance of today's "Average Computer".

As today's average computer is more powerful than yesterday's then the credit's awarded decrease, so that today the average computer gets the same RAC as what yesterdays average computer was awarded.

It's a pretty screwy system, but then again it matches my personal view of the person that thought it up.

Bring back "Flop Counting"

If it matched the original definition of the "Cobblestone" as being based on the number of floating Point operations in a given time period by a theoretical processor then we wouldn't see the vast variation in credit awarded for a given run-time. What happens is the server takes a guess at the number of Floating Point Operations required to complete a work unit, then scales it by the apparent processing rate of the current host, and then applies a couple of fudge factors to try to compensate for the differences in processing time between the hypothetical and real times. But these take no notice of the fact that three things have happened over the years, first processors have become much faster, GPUs have become even faster still, and there have been some big steps in the way the calculations have actually been done, both at the silicone level (improved instruction sets) and the application (optimised applications for example); add to that a sort of feed-forward compensator that tries to predict what the processor will do next time around and we have the mess that is Credit New/Screw. In reality if the credit was awarded purely on the basis of the initial guess the situation would be much fairer, but one would have to take some action to cope with error-tasks (blank or noisy data). It is very noticeable that the majority of projects have implemented their own credit system, based either on a flat rate, or their own scheme as they recognise that any scheme reliant on a property of the host rather than the task is going to fail in some way or other.

---

Bob Smith
Member of Seti PIPPS (Pluto is a Planet Protest Society)
Somewhere in the (un)known Universe?

Seti Classic work units xx WU's = WCG results returned
Seti Classic CPU time xx hrs = WCG total run time

Start to finish the award structure for this project has been badly implemented.
2 things strike me with regards to WCG.
Points are allocated to each completed task.
IIRC, didn't WCG have its own software along the lines of F@H & moved to Boinc later?
Points are still generated but to cater for Boinc a factor of 7 is used.
Something which is accepted.