Stevie,
With ubuntu, things are a little different. When you're running Windows, you do need to go download lots of drivers from your hardware manufacturer; or more often, they come pre-loaded with Windows. With ubuntu, the install process detects your hardware and installs either a community-developed driver or a driver from the manufacturer. Either way, the concept of "downloading a driver" doesn't usually apply unless you have some really specialized hardware from a particularly linux-friendly vendor.
As for the scrolling lag, I would guess Firefox is a bit much for the Aspire One's teensy processor and RAM. I would recommend Google Chrome or Chromium (Chrome is more stable at this point, even though they say it's still a "Developer Preview"); they have much better memory handling than the current version of Firefox. Also, you may try disabling visual effects (compiz) from your System->Appearances window.
If the desktop is still being laggy, you can set it to low graphics mode by pressing Alt+F2 and typing "gconf-editor". Locate the folder apps > netbook-launcher, highlight it, and in the window on the right, check the box titled "force_low_graphics". It won't be as flashy, but it won't make things lag either.
Finally, we may be able to help you locate a better driver for your video card if you let us know what kind of card you have. Open a terminal, type "cd ~", then
sudo lshw > lshw.txt
Find the file lshw.txt (it will be in your home folder), and upload it here; that'll give us more info about your hardware.
...or if you know the make and model of your graphics card, just tell us, we'd be happy to point you in the right direction.
Peace, --Jay