Calibrating credit

Message boards : SETI@home Enhanced : Calibrating credit
Message board moderation

To post messages, you must log in.

1 · 2 · Next

AuthorMessage
Josef W. Segur
Volunteer tester

Send message
Joined: 14 Oct 05
Posts: 1137
Credit: 1,848,733
RAC: 0
United States
Message 1160 - Posted: 28 Oct 2005, 21:09:20 UTC

David Anderson's post yesterday in boinc_dev indicates that setiathome_enhanced will soon be able to send fpops_cumulative back for the scheduler to use in calculating claimed credit.

Rattledagger's test sent a value for fpops_cumulative of 18792566726536. For that, the scheduler claimed 21.75 credits. The 1.88e13 test value is higher than the ~1.76e13 final value for the 0.443 angle range units I've seen, so presumably represents a normal 0.417 - 0.426 angle range.

The BOINC credit system was intended to grant 100 credits per day to a mythical reference computer system which benchmarked at 2000 MIPS for the total of whetstones + dhrystones. The main project seems to be in that range, but setiathome_enhanced will either need to send a larger fpops_cumulative value or the scheduler calculation will need to be adjusted to get into that range. My guess is a typical _enhanced unit deserves about 240 claimed credit. My P4 1.6 GHz system total benchmark is about 2373, suggesting it should earn about 118 credits a day while crunching undisturbed. For the 49.2 hours 4.09 seems to be aiming for with a 0.443 AR unit, I'd expect something like 220 credits.

I wish the term used had been cumulative_work_done rather than cumulative_fpops. For this system of credits to work, optimized applications need to return the same values as unoptimized.
ID: 1160 · Report as offensive
Josef W. Segur
Volunteer tester

Send message
Joined: 14 Oct 05
Posts: 1137
Credit: 1,848,733
RAC: 0
United States
Message 1169 - Posted: 4 Nov 2005, 3:00:42 UTC

As a followup, I'll note that I finished another result today and reported it with BOINC 5.2.6. My claimed credit is 60.67 for 47.5 hours of work, the other participant who has returned that result using BOINC 5.2.5 claimed 252.06.

ID: 1169 · Report as offensive
EdwardPF
Volunteer tester

Send message
Joined: 8 Sep 05
Posts: 82
Credit: 545,522
RAC: 0
United States
Message 1170 - Posted: 5 Nov 2005, 0:09:00 UTC
Last modified: 5 Nov 2005, 0:12:01 UTC

I'm running 5.2.5 and seti 4.09.

120,000 sec's of cpu requests 280 credits ... yours seem low to me ...(???)


ID: 1170 · Report as offensive
Winterknight
Volunteer tester

Send message
Joined: 15 Jun 05
Posts: 709
Credit: 5,834,108
RAC: 0
United Kingdom
Message 1171 - Posted: 5 Nov 2005, 0:27:04 UTC

Looking at the only unit I have done using 4.09 (with 5.2.5) I claimed 251 credits (86,468secs = 1day approx) and Tetsuji claimed 75 credits. I assume he is using 5.2.6 as looking further his result is actually giving figures for the triples/gausians etc.

It would appear that that compared to my running of Seti or Einstein that 250 seems slightly low for a days crunching and that they need to revise the calculation method using SetiB and 5.2.6.
ID: 1171 · Report as offensive
Josef W. Segur
Volunteer tester

Send message
Joined: 14 Oct 05
Posts: 1137
Credit: 1,848,733
RAC: 0
United States
Message 1173 - Posted: 5 Nov 2005, 18:29:26 UTC - in response to Message 1170.  
Last modified: 5 Nov 2005, 18:30:27 UTC

EdwardPF:
I'm running 5.2.5 and seti 4.09.

120,000 sec's of cpu requests 280 credits ... yours seem low to me ...(???)

True, if I had reported the result with 5.2.5 the claim would have been 237.44.

Winterknight:
Looking at the only unit I have done using 4.09 (with 5.2.5) I claimed 251 credits (86,468secs = 1day approx) and Tetsuji claimed 75 credits. I assume he is using 5.2.6 as looking further his result is actually giving figures for the triples/gausians etc.

Tetsuji is using both customized BOINC core clients and customized setiathome_enhanced builds. As he suggested in the 4.09 announcement, we should simply ignore his claims.
It would appear that compared to my running of Seti or Einstein that 250 seems slightly low for a days crunching and that they need to revise the calculation method using SetiB and 5.2.6.

Unless Tetsuji and Eric really think they'll be able to make 4.next 4 times faster...

ID: 1173 · Report as offensive
Pepo
Volunteer tester
Avatar

Send message
Joined: 16 Jun 05
Posts: 172
Credit: 251,583
RAC: 0
Slovakia
Message 1245 - Posted: 1 Dec 2005, 22:17:27 UTC

On 2005-11-06 04:11:44 I started to crunch 437112 on my host 715 with BCC 4.43 or so. The result was continually happily crunched... day after day... Then as the 5.2.x should be able to report cumulative_fpops, I was wondering "what when I reinstall Boinc in the middle of this WU, will it report few hundreds plus a part of 60 credits"?

Shortly (2 hours) before being finished (I was just not aware of, I forgot it) I installed the newest 5.2.13. And now as the result is finally crunched, I noticed it took 237,406.49 seconds (nearly 66 hours), but claimed only 61.60 credits. I understand, it's like summing apples and tomatos, but... if the former calculation would be thrown away, I would expect much less than 2 credits according to much less than 2 hours of accumulating fpops.

Who can explain this? (Seems like randomly assigned 60.xx credits value on finish :-)

Peter
ID: 1245 · Report as offensive
Ingleside
Volunteer tester

Send message
Joined: 14 Jun 05
Posts: 94
Credit: 147,582
RAC: 0
Norway
Message 1246 - Posted: 1 Dec 2005, 22:59:13 UTC - in response to Message 1245.  

Shortly (2 hours) before being finished (I was just not aware of, I forgot it) I installed the newest 5.2.13. And now as the result is finally crunched, I noticed it took 237,406.49 seconds (nearly 66 hours), but claimed only 61.60 credits. I understand, it's like summing apples and tomatos, but... if the former calculation would be thrown away, I would expect much less than 2 credits according to much less than 2 hours of accumulating fpops.

Who can explain this? (Seems like randomly assigned 60.xx credits value on finish :-)


The Seti_Enhanced-application happily adds-up the flops as you're crunching, and only when finished does it tell the core client "the flops-count for this result is 123456789..."

But, before v5.2.6, the core client acts as it didn't hear the message about flops-count, and therefore doesn't report it back to scheduling-server.
ID: 1246 · Report as offensive
Pepo
Volunteer tester
Avatar

Send message
Joined: 16 Jun 05
Posts: 172
Credit: 251,583
RAC: 0
Slovakia
Message 1255 - Posted: 3 Dec 2005, 1:27:15 UTC - in response to Message 1246.  


The Seti_Enhanced-application happily adds-up the flops as you're crunching, and only when finished does it tell the core client "the flops-count for this result is 123456789..."

Soooo simple...
Thanks.
ID: 1255 · Report as offensive
Josef W. Segur
Volunteer tester

Send message
Joined: 14 Oct 05
Posts: 1137
Credit: 1,848,733
RAC: 0
United States
Message 1329 - Posted: 10 Dec 2005, 3:25:11 UTC

Initial note: I'm not personally concerned with credits, but obviously many users are. Setiathome_enhanced is the first application with a stable credit system, I just want it to be as good as possible.

4.09 multiplied the flops estimate by 3 when reporting fpops_cumulative to BOINC. 4.11 multiplies by 9 instead, so claims will be 3 times higher. That's an improvement, but I suspect it is not enough to reach parity with other projects. It gives something like 90 credits per day of crunching on my system which should be near 118 credits (based on the definition of cobblestones from the original 'reference' computer).

The 417 results I've completed and reported for setiathome have averaged 24.6 credits. Average time per unit has been 2 hours 36 minutes, but most have been with an optimized client. Adjusting for the 2.1 ratio (on the reference unit) between that and the standard 4.18 client gives 5 hours 28 minutes as the average time had I been using 4.18. That would give 4.39 units per day, multiplying by 24.6 gives just about 108 credits per day. Others have come up with slightly lower average credits granted per unit, but it would have to be 20.5 to add up to only 90 credits per day for this system.

OTOH, I haven't crunched for many projects and perhaps the fairest interpretatation is to average the credits per day for all projects and target that instead of the original theoretical. Anybody have enough data to suggest whether 4.11 is near that?

ID: 1329 · Report as offensive
Winterknight
Volunteer tester

Send message
Joined: 15 Jun 05
Posts: 709
Credit: 5,834,108
RAC: 0
United Kingdom
Message 1330 - Posted: 10 Dec 2005, 3:58:58 UTC

The latest one I've crunched, but not uploaded yet, has claimed of 147.nn for the other two hosts that have managed to upload/report. Assuming that mine gets similar and it took 18.6 hrs to crunch, 24 / 18.6 * 147 = 190 cobblestones/day.

On normal Seti this computer did 2h:15m using unoptimised, which would mean 10.66 units a day, using your average figure of 24.6 * 10.66 = 262 cobblestones/day.

So I think it needs another 33% to be in line, don't know how 4.09 which awarded 63, squares with 4.11 awarding 147, its not a factor of 9 / 3 = 3.

147 (4.11 claimed) / 63 (4.09 claimed & awarded) = 2.33

2.33 * 1.33 (from percentage increase i think it needs)= 3 ish

Think the arithmetic got screwed somewhere along the line, these things happen and can be re-worked.
ID: 1330 · Report as offensive
Josef W. Segur
Volunteer tester

Send message
Joined: 14 Oct 05
Posts: 1137
Credit: 1,848,733
RAC: 0
United States
Message 1334 - Posted: 10 Dec 2005, 17:57:02 UTC - in response to Message 1330.  

..don't know how 4.09 which awarded 63, squares with 4.11 awarding 147, its not a factor of 9 / 3 = 3.

147 (4.11 claimed) / 63 (4.09 claimed & awarded) = 2.33

2.33 * 1.33 (from percentage increase i think it needs)= 3 ish

Think the arithmetic got screwed somewhere along the line, these things happen and can be re-worked.

The difference is due to angle ranges.

4.09 WUs from the 03oc03aa tape were mostly in the 0.440 to 0.445 range.
4.11 WUs created 2 Dec. from the 13au01aa tape were around 0.567.
4.11 WUs created 5 Dec. from the 05jl01ab tape are about 0.426.

Multiplying the 4.09 claims by three gives about 189 for 0.440 WUs, that seems about in line with 147 for 0.567 units using 4.11. The ratio of CPU time to claim is what's really important.

ID: 1334 · Report as offensive
Winterknight
Volunteer tester

Send message
Joined: 15 Jun 05
Posts: 709
Credit: 5,834,108
RAC: 0
United Kingdom
Message 1336 - Posted: 11 Dec 2005, 0:37:42 UTC - in response to Message 1334.  

The difference is due to angle ranges.

4.09 WUs from the 03oc03aa tape were mostly in the 0.440 to 0.445 range.
4.11 WUs created 2 Dec. from the 13au01aa tape were around 0.567.
4.11 WUs created 5 Dec. from the 05jl01ab tape are about 0.426.

Multiplying the 4.09 claims by three gives about 189 for 0.440 WUs, that seems about in line with 147 for 0.567 units using 4.11. The ratio of CPU time to claim is what's really important.


Of course, I didn't even look at crunch time or range_angles.

(must remember to think, BEFORE, opening mouth}

ID: 1336 · Report as offensive
Josef W. Segur
Volunteer tester

Send message
Joined: 14 Oct 05
Posts: 1137
Credit: 1,848,733
RAC: 0
United States
Message 1343 - Posted: 11 Dec 2005, 22:37:30 UTC

FWIW, here's some more data.

My system finished its first 0.426 WU created 5 Dec. this morning, with final_cpu_time=212401.421 and fpops_cumulative=165501054321498. When I'm able to get it uploaded and returned, the claim will be 191.55 credits [the formula is (fpops_cumulative/1e9)*(100/86400)]. Earlier this morning BOINC (standard 5.2.12) ran the automatic benchmarks, getting 808.244 Whetstone MIPS and 1522.598 Dhrystone MIPS. Without the fpops_cumulative, the claim would be 286.50 credits. 191.55/286.5 = 0.6686.

Note: Although Ingleside noted 0.417 as "the most common angle-range", I believe that was true only when the splitter was giving angle range as the great circle distance between the starting and ending RA and Dec. The more accurate true_angle_range in current WUs based on all of the coordinates seems to give about 0.426 most commonly.

I also gathered some data for other projects (Einstein, LHC, Predictor, and Rosetta). The method was to go to the "Top Participants" list and find unhidden computers with at least 10 recent results. I calculated the cobblestones/day rating for each host ((Whetstone MIPS + Dhrystone MIPS)/20) and the granted credits/day ((sum of granted credits/sum of times)/(24*60*60)). I only did two hosts for each project and don't know how much the benchmarks vary for those hosts. Anyhow, I came up with granted credit to cobblestone rating ratios ranging from 0.75 to 1.14, average 0.94.
                                                    Joe

ID: 1343 · Report as offensive
Winterknight
Volunteer tester

Send message
Joined: 15 Jun 05
Posts: 709
Credit: 5,834,108
RAC: 0
United Kingdom
Message 1347 - Posted: 12 Dec 2005, 6:32:11 UTC
Last modified: 12 Dec 2005, 7:24:14 UTC

I've managed to complete two 4.11 units and upload/report.

The first took 66995.36 seconds to complete, claimed credit 147.85, others who have returned unit 147.80 and 147.15, therefore granted 147.80. This equates to approx 190 credits/day.

On the old system, benchmark * time/ 20 days, would have been claim of 198.40. or approx 255/day.

The second took 104213.73 seconds to complete, claimed credit 191.62. This equates to approx 159 credits/day.

On the old system, benchmark * time/ 20 days, would have been claim of 308.63 or approx 255/day

On normal Seti, using standard application, I would average 10.5 units per day, with guessed average of 24 granted credits/day this would be 252 credits/day, fairly close to old system calculation.

Therefore assume that, and taking Josef's post into account, that the calculation for FLops credits system needs to be increased by about 33%.

P.S. Claimed credit for Einstein same computer 84.5 credits * 3 units /day = 253.5

Andy
ID: 1347 · Report as offensive
Winterknight
Volunteer tester

Send message
Joined: 15 Jun 05
Posts: 709
Credit: 5,834,108
RAC: 0
United Kingdom
Message 1617 - Posted: 12 Jan 2006, 8:45:30 UTC

At the moment, and presumably when this becomes mainstream Seti, it is intended to keep the present claimed/granted credits at approx 190 for units with most common angle_range. It has been suggested by me and others that this seems a little low.

Therefore I have to ask, has this version crunched the standard seti reference unit. And if so, did the claimed credits for that equate to that claimed for the present Seti application, which is reported to give 100 cobblestones/day for the hypothetical reference computer?

I (we) would like to see some figures to substanciate the present calculation please.
ID: 1617 · Report as offensive
Honza
Volunteer tester

Send message
Joined: 19 Jun 05
Posts: 42
Credit: 9,057
RAC: 0
Czech Republic
Message 1618 - Posted: 12 Jan 2006, 10:53:09 UTC

This is not only problem of SETI.
trux has compiled his new version of BOINC core, which - among other new functions still missing in regular core - has calibration feature.

So, the general question for me is: should each project deal with claimed/granted issue (partially by implementation of FLOPs) or should this solve BOINC core?
ID: 1618 · Report as offensive
Josef W. Segur
Volunteer tester

Send message
Joined: 14 Oct 05
Posts: 1137
Credit: 1,848,733
RAC: 0
United States
Message 1892 - Posted: 23 Jan 2006, 5:33:44 UTC

Some additional thoughts.

1. On the subject of Trux's core client, his method of adjusting credit uses a filtered and delayed version of Duration Correction Factor. That immediately makes the method unusable for any project with multiple applications or an application which does several different kinds of work.

2. I started a separate thread on Plots which contains information pertinent here. To summarize, credit calibration should be heavily weighted so that WUs near 0.436 AR get the right amount of credit per time.

3. From the S@H host.xml data shown at BOINCstats, slightly over 88% of hosts for the project are running some version of Windows, Linux has slightly less than 8%, all others comprise the remaining 4%. That suggests that the calibration should be based mostly on the Windows build.

4. Figures for my system strongly support the suggestion by Andy (WinterKnight) that the fpops_cumulative value should be increased another 33%. Changing the multiplier from its present 9 to 12 would be just about right.
                                                         Joe
ID: 1892 · Report as offensive
Profile Steve Cressman
Volunteer tester
Avatar

Send message
Joined: 14 Nov 05
Posts: 296
Credit: 13,874
RAC: 0
Canada
Message 1898 - Posted: 23 Jan 2006, 17:59:47 UTC - in response to Message 1892.  


4. Figures for my system strongly support the suggestion by Andy (WinterKnight) that the fpops_cumulative value should be increased another 33%. Changing the multiplier from its present 9 to 12 would be just about right.
                                                         Joe


I agree and that backs up what I posted in another thread. My rough calculations showed that with a multiplier of 9 we only get ~60% of what we would for the same amount of time on the other projects including seti v4.18. A multiplier of 12 should bring us back on par.
98SE XP2500+ @ 2.1 GHz Boinc v5.8.8
ID: 1898 · Report as offensive
Profile Saenger
Volunteer tester
Avatar

Send message
Joined: 13 Jan 06
Posts: 8
Credit: 53,651
RAC: 0
Germany
Message 1933 - Posted: 25 Jan 2006, 23:14:26 UTC

I just had my first through (nothing granted so far, but I expect the usual 190.
That's OK within this project, but not compared to others (exception: Folding Beta), as I would usually claim double that amount, and I don't run an optimised client, plain and simple stock (OK, not entirely right, it's the 5.3.9 form CPDN, but no benchmark adjustment)
Runtime: 147094.14
Benchmarks: 1627 + 2755
Should-be claimed Credit: 373
Claimed Credit: 190
Factor: 1.96
Grüße vom Sänger
ID: 1933 · Report as offensive
Josef W. Segur
Volunteer tester

Send message
Joined: 14 Oct 05
Posts: 1137
Credit: 1,848,733
RAC: 0
United States
Message 2076 - Posted: 30 Jan 2006, 23:56:32 UTC

The table below shows data gathered from the first 3 pages of "top computers" on Jan. 29. The hosts listed are AFAICT not running an optimized version of setiathome_enhanced, and each had one or more results for WUs close to 0.426 angle range (190 to 192 credits).

The c/day column is the host cobblestone rating, (whetstone MIPS + dhrystone MIPS) / 20, the same formula which gives 100 for the "reference computer". The e/day column is (credits from enhanced / time in seconds ) * 86400 for the near 0.426 normal unit.

    Host |c/day |e/day |Sys type
    ---- |----- |----- |---------------
    3665 |129.8 |118.3 |(Windows Intel)
    3970 |144.3 |109.9 |(Windows Intel)
    3925 |379.7 |158.0 |(Windows Intel)
    3088 |334.6 |119.6 |(Windows Intel)
    3719 |125.4 |111.3 |(Windows Intel)
    4045 |201.3 |140.7 |(Windows Intel)
    2913 |206.8 |122.4 |(Windows Intel)
    3778 |146.1 |100.2 |(Windows Intel)
    3686 |275.9 |129.5 |(Windows Intel)
    4027 |145.9 |110.3 |(Windows Intel)
    2605 |118.6 |080.2 |(Windows Intel)
    3723 |259.6 |124.5 |(Windows Intel)
    3387 |124.0 |104.1 |(Windows Intel)

    4134 |286.3 |148.4 |(Windows AMD)
    3726 |694.8 |755.6 |(Windows AMD)
    4042 |424.9 |133.8 |(Windows AMD)
    2983 |273.4 |137.8 |(Windows AMD)
    4011 |440.7 |141.0 |(Windows AMD)
    3917 |634.9 |159.1 |(Windows AMD)
    3915 |728.2 |151.5 |(Windows AMD)
    4012 |471.1 |146.1 |(Windows AMD)
    3615 |206.2 |119.2 |(Windows AMD)
    3664 |252.8 |132.3 |(Windows AMD)
    2806 |388.4 |197.0 |(Windows AMD)

    3623 |094.5 |195.9 |(Linux Intel)
    3960 |141.2 |196.8 |(Linux Intel)
    4316 |132.0 |248.4 |(Linux Intel)
    3020 |059.3 |163.0 |(Linux Intel)
    4133 |082.3 |185.4 |(Linux Intel)
    1460 |050.0 |222.6 |(Linux Intel)
    3371 |066.3 |177.0 |(Linux Intel)
    2848 |088.0 |192.8 |(Linux Intel)

    3442 |112.2 |201.2 |(Linux AMD)
    3444 |112.3 |210.5 |(Linux AMD)
    3797 |138.4 |228.1 |(Linux AMD)
    2172 |140.4 |250.0 |(Linux AMD)
    2089 |133.4 |242.9 |(Linux AMD)
    3662 |160.3 |315.5 |(Linux AMD)
    2036 |147.6 |263.1 |(Linux AMD)


Note: I didn't screen for hosts running an optimized BOINC to increase benchmarks, nor do I know which of those hosts have unstable benchmarks.

My thoughts:

1. The Windows data suggests that even our previous suggestion of a 33% increase is quite conservative.

2. Only part of the Linux difference can be accounted for by the known tendency of BOINC for Linux to give low benchmarks. I think the Linux build of 5.02 is more efficient than the Windows build. That's fine, enhanced will compensate Linux participants for previous low credit claims due to the low benchmarks.

                                               Joe
ID: 2076 · Report as offensive
1 · 2 · Next

Message boards : SETI@home Enhanced : Calibrating credit


 
©2022 University of California
 
SETI@home and Astropulse are funded by grants from the National Science Foundation, NASA, and donations from SETI@home volunteers. AstroPulse is funded in part by the NSF through grant AST-0307956.