After I finally sold my sony vaio I got myself a dell inspiron 1520. All seemed to work fine after I installed a Linux Gentoo but on the first big download I noticed the error in the title, notable transfer timeouts, sudden deaths and if none of the above happened I would still get a very low transfer limit (maxed to 200KBps).
Below is a link to a linux kernel config that has b43 driver fully working (my wifi card is a bcm4311, arch is x86_64). What I think it was happening is for some reason I was using PIO mode instead of DMA mode. Normally PIO mode is enabled for debugging purposes and I think somehow I enabled it (or there was a FORCE_PIO enabled by default? I don't know). I'm not 100% sure that this was the problem as I didn't had time to fully check (yet) but a diff on the config files shows it clearly.
diff -Nu /usr/src/linux-2.6.31-gentoo/.config /usr/src/linux/.config | grep PIO
-# CONFIG_B43LEGACY_PIO_MODE is not set