Andy,

Could you edit in the BOINC versions to that comparison? I believe BOINC benchmarking code, and hence BM results, may have changed over the eons.

Not by a facter of more than double, though. What compiler are they using? Not the naughty Intel one with the AMD cripplecode, I hope.

...
Resource share for project is a number. Projects have no indication of resource share of any other projects host is attached to.
...

The core client reduces the shares to a fractional value. Because my hosts have equal shares for Seti main and here, in sched_request it's <resource_share_fraction>0.500000</resource_share_fraction>. But as you noted in an earlier post, the Scheduler does not use that value when determining whether a WU is feasible for the host. The comment block preceding the time estimate function says:

// estimate the amount of real time to complete this WU,
// taking into account active_frac etc.
// Note: don't factor in resource_share_fraction.
// The core client no longer necessarily does round-robin
// across all projects.

                                                                Joe

Richard,
I didn't include any further details, BOINC ver etc, because I wanted it to reflect what is reported to project by the host now, not a guess if the owner updated BOINC. Which I have not done, and although not certain for this host but I know Al is BOINC Alpha tester.

Joe,
Didn't know <resource_share_fraction> was still there, thought it had been removed when JM7 went to debts to determine downloads. <resource_share_fraction> would have to be a very dynamic number, which it isn't, because it should need to take into account NNT's and if other projects the host is attached to actually have tasks.

As an attempt to see how the WMIPS fpops benchmark correlates with actual productivity, I decided to try a comparison of that benchmark to the credit/time ratios. I fetched the host xml for this project, eliminated hosts which either had not contacted the servers within the last 30 days or had RAC below 5. That left 2676 active hosts, here's a plot:

There's probably a fair amount of bad data included, but both sets of data adapt rapidly so it should be decent overall.

What I see is that an overall "best fit" straight line would have a significantly different slope than a similar "best fit" for just the hosts which may be marginal for doing AP work. Another thing is the huge range of actual credit rates for a given WMIPS, actual productivity varies over at least a 6:1 range for any benchmark value. Finally, that data set includes 30 hosts with credit_per_cpu_sec below the 0.000721 rate which corresponds to 1869.04 credits in 30 days. Those 30 hosts have WMIPS ratings ranging from 108 to 1433.

If I had broadband I'd do the same sort of plot for SETI main hosts, but the file is over 300 MiB and I think it would take more than a day to download. I don't know exactly what would happen when the daily update replaced the file, at best my in_progress download would force the server to keep the old file for awhile, at worst the download would abort.

                                                                 Joe

I don't recommend going for the main hosts file, even with broadband - that 300MB download expands to at least 2GB of raw XML! But I have a copy now, and I've managed to reproduce your recipe for the chart (you didn't mention that you had to throw out some hosts with zero credit/CPUsec, to get down to 2676 hosts).

Processing an XML that size is going to take some ingenuity with the tools at hand, but I'll see if I can do an equivalent chart for Main.

I don't recommend going for the main hosts file, even with broadband - that 300MB download expands to at least 2GB of raw XML! But I have a copy now, and I've managed to reproduce your recipe for the chart (you didn't mention that you had to throw out some hosts with zero credit/CPUsec, to get down to 2676 hosts).

Processing an XML that size is going to take some ingenuity with the tools at hand, but I'll see if I can do an equivalent chart for Main.

Thanks for tackling this, and I apologize for not keeping better track of the filtering I did.

                                                                Joe

I don't recommend going for the main hosts file, even with broadband - that 300MB download expands to at least 2GB of raw XML! But I have a copy now, and I've managed to reproduce your recipe for the chart (you didn't mention that you had to throw out some hosts with zero credit/CPUsec, to get down to 2676 hosts).

Processing an XML that size is going to take some ingenuity with the tools at hand, but I'll see if I can do an equivalent chart for Main.

Thanks for tackling this, and I apologize for not keeping better track of the filtering I did.

                                                                Joe

I think I've been able to work out a way of doing it - short tests are working, and I've got a core working on the main file now. I'm generating a CSV file with

HostID, RAC, p_vendor, os_name, p_iops, p_fpops, credit_per_cpu_sec

- it zips up the Beta run to about 50KB. I suspect the main file will be much, much bigger, but I can filter it more strictly here and email it to you to analyse. Current limits are RAC>5 (as before), rpc contact since the beginning of July.

Edit - I was afraid of that. 277,130 records, 5.6MB zipped. Also, I can't chart it like that (Excel will only accept 64K rows). What would be the best way of filtering without distortion?

...
I was afraid of that. 277,130 records, 5.6MB zipped. Also, I can't chart it like that (Excel will only accept 64K rows). What would be the best way of filtering without distortion?

The 16-bit version of Excel I have only accepts 16K rows, and for graphing won't deal with more than 4K X values. Opening a csv file with more than 16K rows makes multiple sheets each with 16K rows, so it's still possible to work with larger data sets, except it's on my Win95 system with only 128 MiB RAM.

I'm not sure a plot with 270K points would be much good anyhow, but I'd set up bins for credit rate each containing all the corresponding WMIPS values in multiple columns.

The GS-Calc spreadsheet program can handle 2M rows. I have the 6.0 version (registered) on my daily driver Win98 system, 7.x requires Win2k or above. It's $20 to buy, 30 day free evaluation with crippling:

· Limited export/import functions (up to 15 rows)
· Printing the first page only

I'm not sure, but think loading a csv file counts as "import", and the plotting capabilities aren't as good as Excel.

I would definitely like a copy of the 5.6M archive, just cleared out my email so there shouldn't be any problem. OTOH, if you put the file somewhere I could download it without the 30% or more overhead of email Base64/uue encoding that would be even better.

                                                              Joe

...
I would definitely like a copy of the 5.6M archive, just cleared out my email so there shouldn't be any problem. OTOH, if you put the file somewhere I could download it without the 30% or more overhead of email Base64/uue encoding that would be even better.

                                                              Joe

Sorry, I don't have any web/ftp space set up, so email will be quicker. It's on its way.

...
I would definitely like a copy of the 5.6M archive, just cleared out my email so there shouldn't be any problem. OTOH, if you put the file somewhere I could download it without the 30% or more overhead of email Base64/uue encoding that would be even better.

                                                              Joe

Sorry, I don't have any web/ftp space set up, so email will be quicker. It's on its way.

Thanks, I see it on the server just about the same size as an AP WU so just fine.

{edit}And I now have the file on disk. Looks fine in UltraEdit32 though GS-Calc ran out of memory while trying to parse it. 512 MB may or may not be enough, I'll check with other stuff shut down shortly.{/edit}

                                                                 Joe

FWIW, here's a plot from that S@H main data:


I did remove 52 pairs where the WMIPS value was 7000 or above, probably anything above 5000 is bogus and they would have put a huge amount of near-empty space at the top of the plot (the highest was nearly 867000). Obviously I should have done the same for credits_per_cpu_second above 0.03 or so but didn't realize it soon enough. I binned the credits_per_cpu_second at 0.00014 intervals to give 500 possible X values and the WMIPS at 20 intervals to give 350 possible Y values.

From the data set, there were 6220 hosts with credits_per_cpu_second values less than the 0.000721 equivalent to doing an AP WU in 30 days of CPU time, those hosts had WMIPS up to 2008.3.

                                                              Joe

And also FWIW, here's a more detailed view of the same data:


(direct link)

To make the numbers manageable, I took just the last hour of server contacts from the hosts file, and filtered to the limits Joe suggested (7000 WMIPS, 0.03 CR / CPU sec): that gave me

Intel/Windows        22,174
AMD/Windows           6,856
Intel/Linux           1,899
AMD/Linux               713
Intel/Darwin          1,944
PPC/Darwin              607

I then scaled the graph to show only 4000 WMIPS / 0.02 CRsec, because it was a bit sparse round the edges.

I think we can tell who the optimised app users are, and how few of them!

Interesting plots guys.
The coloured plot showing difference in AMD/Intel is wider than I thought it would be. Shows that any policies taken using tests on only one cpu makers cpu's would skew overall results.

Assuming my computer reported during that hour I'm pretty sure I know which is mine, having worked out cr/sec, and knowing WMIPS figure.

Interesting plots guys.
The coloured plot showing difference in AMD/Intel is wider than I thought it would be. Shows that any policies taken using tests on only one cpu makers cpu's would skew overall results.

Assuming my computer reported during that hour I'm pretty sure I know which is mine, having worked out cr/sec, and knowing WMIPS figure.

I can label the dots with Host IDs if you want.....

We can only speculate on the processor differences. Remember, we're plotting BOINC benchmark vs scientific output, so the AMDs might either have a low scientific output, or be really good at running the BOINC benchmark. We know that BOINC/Linux tends to benchmark low, which presumably accounts for the shallower slope of the turquoise/yellow trend compared with the pink/blue trends.

We can only speculate on the processor differences. Remember, we're plotting BOINC benchmark vs scientific output, so the AMDs might either have a low scientific output, or be really good at running the BOINC benchmark. We know that BOINC/Linux tends to benchmark low, which presumably accounts for the shallower slope of the turquoise/yellow trend compared with the pink/blue trends.

AMD CPUs have larger L1 cache, perhaps the benchmark as implemented by BOINC doesn't quite all fit in Intel L1 caches. There's also a possibility that the particular mix of work being reported in that time period had an effect.

One of the curious things I noticed in the Beta data was that there was a tendency for the Whetstone benchmarks to cluster at some values rather than having an even random variation. To a certain extent that might be simply from identical CPUs clocked identically, but I noted one group where there were a dozen or more benchmark values identical to 10 digits. It probably doesn't signify anything other than the way the 8 subtests of the benchmark are combined, but it was interesting to see the similar horizontal line effect in your plot.

                                                                  Joe

First draft was written some time ago.

Here's the faq ... its structure comes from Winterknight's post. (Thanks!) Let me know if you see any errors. Also, I am in need of an app_info.xml example. Thanks,

J

.... Also, I am in need of an app_info.xml example. Thanks,

J

Could you give us the definitive application names, executable file list, and version numbers as they will be launched on Main, please, and we'll roll a minimalist sample for you (the current versions floating around in parallel threads are a bit wordy). NB I'm offering for Windows only: you'll need a Linux, and eventually a Mac, sample too.

The other thing you'll need to post is an accessible manual download location for the files. Historically, it's been relatively easy to download Beta apps as needed, but difficult to get hold of SETI ones (browser http downloads got routed through a different, expensive, campus network - which got blocked for cost reasons). I suggest you consult Eric about this.

.... Also, I am in need of an app_info.xml example. Thanks,

J

... NB I'm offering for Windows only: you'll need a Linux, and eventually a Mac, sample too....

O.k. i volunteer for the linux version, when the needed information are available.

---

_\|/_
U r s

Could you give us the definitive application names, executable file list, and version numbers as they will be launched on Main, please, and we'll roll a minimalist sample for you (the current versions floating around in parallel threads are a bit wordy). NB I'm offering for Windows only: you'll need a Linux, and eventually a Mac, sample too.

The applications are the same as for beta 4.35, i.e.

ap_graphics_4.35_i686-pc-linux-gnu
ap_graphics_4.35_windows_intelx86.exe
ap_graphics_4.35_x86_64-pc-linux-gnu
astropulse_4.35_AUTHORS
astropulse_4.35_COPYING
astropulse_4.35_COPYRIGHT
astropulse_4.35_i686-pc-linux-gnu
astropulse_4.35_windows_intelx86.exe
astropulse_4.35_x86_64-pc-linux-gnu

The other thing you'll need to post is an accessible manual download location for the files. Historically, it's been relatively easy to download Beta apps as needed, but difficult to get hold of SETI ones (browser http downloads got routed through a different, expensive, campus network - which got blocked for cost reasons). I suggest you consult Eric about this.

Eric says:

http://boinc2.ssl.berkeley.edu/sah/download_fanout/

That's cool stuff. You can really tell that there aren't a lot of people out there running MacOS X on non-apple hardware, because the benchmark results cluster really well.

It's hard to ascribe the differences to a specific aspect of the processor and decide whether the benchmarks are different between the processors or the efficiency with which they run the application is.

One of the hazards of using FLOP counting for credit is that credit per cpu second might not match very well with the benchmarks for specific machines, and perhaps people with machines that get too little credit will migrate to other projects.

The scheduling aspect of this is somewhat annoying, but BOINC scheduling has always been that way. We've made a change to the scheduler such that a fudge factor is applied to astropulse deadlines when determining whether a host could complete them in time. Our current fudge factor is 0.77 If we set a 14 day deadline, no result will be sent if a host would not normally be able to compute the result in 10.75 days. If that's not enough of a fudge, we can decrease it.

---