Curiously Compressible Data

Message boards : AstroPulse : Curiously Compressible Data
Message board moderation

To post messages, you must log in.

AuthorMessage
Josef W. Segur
Volunteer tester

Send message
Joined: 14 Oct 05
Posts: 1137
Credit: 1,848,733
RAC: 0
United States
Message 33612 - Posted: 28 May 2008, 2:19:10 UTC

I downloaded WU 1144837 earlier today, and was surprised by the rate on my dial-up connection. It was nearly double normal speed, indicating that modem compression was finding a lot of repetitive patterns. So I looked at the data, here's a small sample of hex dump:



There are mostly 0, 2, 8, and A characters. That indicates the second bit of most bit pairs is zero, very unusual. I think the data is damaged beyond any possibility of getting valid science from it, but it may test the application's ability to handle off-nominal cases.

Using ordinary zip compression on that WU reduces it from 8 MB to 4.5 MB. Doing the same on WU 1143119 only shrinks it by 0.2 MB or so.
                                           Joe


ID: 33612 · Report as offensive
Profile Pappa
Volunteer tester
Avatar

Send message
Joined: 13 Nov 05
Posts: 1724
Credit: 3,121,901
RAC: 0
United States
Message 33613 - Posted: 28 May 2008, 2:51:14 UTC

Joe Note sent

Thanks to Paul and Friends
Please consider a Donation to the Seti Project
ID: 33613 · Report as offensive
Hofman's Atlantic
Volunteer tester

Send message
Joined: 22 Feb 06
Posts: 122
Credit: 9,551,779
RAC: 0
United States
Message 33615 - Posted: 28 May 2008, 4:22:36 UTC - in response to Message 33612.  
Last modified: 28 May 2008, 4:49:26 UTC

Joe:
I have three mates to your ap_27fe08ac_B3_P1_00349_20080425_27217.wu

ap_27fe08ac_B3_P1_00006_20080425_27217.wu]
ap_27fe08ac_B3_P1_00073_20080425_27217.wu]
ap_27fe08ac_B3_P1_00077_20080425_27217.wu]

Which also zip to 44%

One of these should start running in 16 hours {4:00 pm EDLST 6/28/08).
Another starts about 5 hours later.

I will keep an eye on them and report anything unusual.

Conrad S.
ID: 33615 · Report as offensive
Richard Haselgrove
Volunteer tester

Send message
Joined: 3 Jan 07
Posts: 1451
Credit: 3,272,268
RAC: 0
United Kingdom
Message 33621 - Posted: 28 May 2008, 8:38:27 UTC

I have ap_27fe08ac_B3_P1_00257_20080425_27217.wu, which is similarly compressible. It's just run for an hour so far, but seems normal.

Like Joe, I got it as a re-issue - WU 1144745 has had one successful run with AP 4.30 already.
ID: 33621 · Report as offensive
vonkorff
Project administrator
Project developer
Project scientist

Send message
Joined: 10 Feb 07
Posts: 84
Credit: 24,876
RAC: 0
United States
Message 33630 - Posted: 28 May 2008, 21:08:07 UTC - in response to Message 33621.  

I have ap_27fe08ac_B3_P1_00257_20080425_27217.wu, which is similarly compressible. It's just run for an hour so far, but seems normal.


All of the problematic workunits are B3_P1 (beam 3, polarization 1) from the same tape, 27fe08ac. I've checked other tapes, and the issue seems to be limited to a specific time period, beam, and polarization.

For instance, one tape from the same time period (march 5) is compressible on B3 P1 but not B3 P0. A tape from April 24 is not compressible on B3 P1, and a tape from March 22 of 2007 is not compressible on B3 P1. A more thorough investigation might be called for.

Let me know if you find any compressible workunits that are not B3 P1.
ID: 33630 · Report as offensive
JLDun
Volunteer tester
Avatar

Send message
Joined: 23 May 07
Posts: 108
Credit: 127,898
RAC: 0
United States
Message 33638 - Posted: 29 May 2008, 7:43:06 UTC - in response to Message 33630.  

Since I'm not quite sure what qualifies as 'compressable' right now:
WU 1143100
WU Name: ap_27fe08ac_B1_P1_00204_20080424_15931.wu
Using: WINZIP 10.0 (Build 7245)
Prezip: 8,392,012 Bytes
Postzip: 8,145,085 Bytes
"3%"

On a WinXP SP2 Celeron M Laptop
ID: 33638 · Report as offensive

Message boards : AstroPulse : Curiously Compressible Data


 
©2021 University of California
 
SETI@home and Astropulse are funded by grants from the National Science Foundation, NASA, and donations from SETI@home volunteers. AstroPulse is funded in part by the NSF through grant AST-0307956.