We all know that eukaryotes are bigger than prokaryotes. On average. Mostly. Of course our pathetic attempts at generalisation are too often devastated in a counterattack by nature's awesomest power: variation. There's variation within species, making it a necessity to ultimately tie biology back to populations from time to time -- but that's a topic for a later time. There's even more variation in higher hierarchical orders, making it rather difficult to say much of anything about a group of organisms, sometimes. But the divide between bacteria and eukaryotes is surely a case where no ambiguity is cast, especially in something as obvious as size!
Of course, we later learn about very large bacteria, and very small protists, and the non-insignificant overlap between the two (this also happens between animals and protists -- Polilov 2011 Arthropod Struc Dev). Ranging from 200-800 microns, Epulopiscium, a bacterial denizen of fish guts, is so massive, it was initially mistaken for a protist (Montgomery & Pollack 1988 J Eukaryotic Microbiol)! To give you a sense of scale, here's a fairly famous image showing the giant next to Paramecium (also quite big for a protist, even) and E.coli reduced to tiny specks in the background.Conversely, eukaryotes can get quite tiny. Micromonas, as its name may suggest, is a tiny green alga that thrives in the oceans. It is so incredibly small that one can only marvel at how the minimal set of vital organelles is crammed in there like in miniature Japanese electronics. A similarly sized critter, Ostreococcus, doesn't have room for extra microtubules to fit in, and does mitosis with fewer 'tubes than chromosomes (recall that in the classic mitotic spindle, each chromosome gets its special bundle of microtubules). Point is, at less than 2 microns, this eukaryote is smaller than many prokaryotes. And there's a large and every-growing group of these 'picoeukaryotes' distributed across many phyla, most still unseen and only known from environmental DNA sequences.
The max-min size ranges for the various protist supergroups, and bacteria, have been compiled into a handy figure from a paper quite relevantly titled Protists are microbes too: a perspective, arguing that mainstream microbiologists(=bacteriologists) need not shun their pets' eukaryotic neighbours based on size alone -- eukaryotes can be no less 'micro' than prokaryotes!
All that said, size does matter, and eukaryotes are largely bigger than prokaryotes. If the above figure were replaced by a box plot, this point would become clearer. This issue shows up a fair bit in discussions of eukaryogenesis, or the origin and early evolution of eukaryotes. For one very basic thing, chomping on bacteria by engulfing them (phagocytosis -- a key ability of eukaryotes) sort of requires the chomper to be bigger than the chompee -- although some protists have a remarkable ability to engulf and devour critters bigger than themselves, presumably by recruiting lots of extra membrane. Being big can also help keep you away from becoming chomped upon yourself. Again, generally: it doesn't help the nematode much as it suffocates in the digestive vacuole of a Vampyrellid amoeba. We can assume that in some circumstances, getting bigger may well be favoured by selection. But then we wonder: why don't bacteria typically get bigger? Instead of getting murdered by the thousands, why can't the Klebsiella in my Paramecium cultures simply get bigger and gleefully watch the ciliates choke on it?
The common explanation is that they simply can't. Physical constrains have [mostly] forever locked the prokaryotic cell within a certain size. It needed to first acquire something special, something a eukaryote has, before its bloated form could even be viable. The special eukaryotic thing can be argued to be, among many others, the mitochondrion, supplying enough energy to maintain the bloated cell, or the cytoskeleton, providing structural support for the cell. Presumably, they both have a role, and perhaps the rarity of large bacteria can be in large part explained by biophysical limitations, both structural and energetic. But some, like our friend in the fish gut, can still get big. What I've been wondering about lately is do bacteria really need to be big?
Perhaps what may constrain bacterial size isn't just biophysical properties, but also 'evolutionary' properties -- namely the presumed trade-off between size and replication rate. Generally, you can't both grow and reproduce simultaneously, and any time you spend making your cell larger is time you spend not dividing. I'm not one for frivolous adaptationist explanations, but this trade-off would have a pretty damn direct effect on fitness: the growth rate would be directly reduced in a population where cells spend more of their time getting big. In other words, if it's real, it will probably matter. Furthermore, a reduction in reproductive rate may shrink your effective population size (crudely skipping a few important factors like carrying capacity), which weakens the relative strength of selection vs. drift, leading to deleterious mutation accumulation, etc. In other words, getting big does not come cheap. And in addition to physical constrains, one must also consider population genetic ones: all other things equal, a competition between individuals from a larger population versus those from a smaller one would probably more often grant victory to those of the larger one. The quantitatively-challenged group would suffer from a bad case of genetic load.
But why, then, don't bacteria that are now ruthlessly hunted in the post-eukaryogenesis world finally have enough selective pressure to grow bigger? Biophysical constraints do play a crucial role in general, but we should also weigh those factors with the evolutionary side of things. Perhaps in the grand scheme of things, the selective pressure by predation just isn't big enough to sacrifice precious replication time for growth. Or maybe it is -- who knows? We can speculate (loudly!) about this for years and years, but speculation can inspire potentially interesting experiments. And yes, you can do experiments in evolution, thankfully. For example, one could try to evolve bacteria, under more or less equal effective population sizes, with and without various predators (small enough that they'd choke on a slightly bigger bacterium). Will size be selected upon? Of course if it isn't, you can't rule out physical constraints. You could try to artificially select for larger bacteria, but that might be a logistic nightmare, given how tiny they are. You could fiddle with their cell cycle machinery to grow larger cells, and compete them against smaller counterparts in the presence or absense of predators. You could then fiddle with effective population sizes... there's plenty of work.
I don't quite have the patience or expertise for these sorts of experiments (and some must have been done already!). The point here was to remind ourselves that the most obvious explanation can sometimes blind us of less obvious, but potentially important, additional factors. It's tempting to accept "biophysical constraints" as the catch-all answer to the issue of why bacteria tend to be small, but perhaps something like population size may be involved as well, among other aspects of biology that I'm ignorant of. And this stuff can be spared from the tumultuous world of pure speculation and tested with real experiments. There's plenty of work to be done!