One of the first things you learn once you start taking biology as a subject is that life is split into two separate domains - prokeryotes and eukaryotes. Prokaryotes are small and blobby and have no nucleus or internal organisation, while eukaryotes are big and multicellular and contain not just a nucleus, but all sorts of other organelles inside the cell such as mitochondria, chloroplasts, vacuoles and exciting things with names like endoplasmic reticulum.
However as you can see in the picture above even though there are no proper membrane-bound organelles in the prokaryote cell that doesn't make it a featureless blob. The diagram above shows a structure labelled the 'nucleoid' which is the area of the cell in which the DNA is kept, tightly coiled up to keep it out of the way of reactions going on inside the cell.
Another structure that has been found in many bacterial cells is a little membrane-bound granule that stores calcium called an acidocalcisome. Acidocalcisomes were first found in single-celled protists but have since been found in a range of bacteria as well as eukaryotic organisms. This suggests that they are a fairly ancient method of storing calcium within a cell. Whether they count as an 'organelle' is slightly more questionable, but they certainly hint at a higher level of intracellular organisation inside prokaryotes.
It's not just calcium stores either, there are also little membrane covered phosphate stores found in bacterial cells. As phosphate is one of the key molecules in energy usage, a phosphate volutin granule is quite an exciting thought. The granule is characterised by its internal acidity, a high electron density and a specialised surrounding membrane. The membrane surrounding the granule contains a high number of phosphate transporters, to allow the store to be filled and emptied as the phosphate is used by the cell.
The interesting thing about both of these structures is that they are found widely in both prokaryotes and eukaryotes which suggests that they evolved before these two lineages diverged. The last common ancestor of both people and bacteria might have had these little granules for storing calcium and phosphate. However, the last-common-ancestor story is confused by the fact that modern eukaryotes all contain ex-bacteria in the form of mitochondria (and chloroplasts in plants). It is possible that the ability to store calcium and phosphate was conferred on the eukaryotes by their prokaryote symbiotes before they became mitochondra.
I personally would support the hypothesis that these granular membrane-enclosed stores were indeed a feature of the last common ancestor of bacteria, archaea and eukaryotes. Highly acidic and electonegative little bundles of phosphate are dangerous thing to keep in close proximity to DNA. It's a sensible survival strategy to lock them up in membrane bound vesicles, releasing them only when necessary, and this survival strategy would have conferred a significant advantage in an ancient microbiological world.
Ref 1 = Seufferheld, M. (2003). Identification of Organelles in Bacteria Similar to Acidocalcisomes of Unicellular Eukaryotes Journal of Biological Chemistry, 278 (32), 29971-29978 DOI: 10.1074/jbc.M304548200
Ref 2 = Seufferheld, M., Kim, K., Whitfield, J., Valerio, A., & Caetano-Anolles, G. (2011). Evolution of vacuolar proton pyrophosphatase domains and volutin granules: clues into the early evolutionary origin of the acidocalcisome Biology Direct, 6 (1) DOI: 10.1186/1745-6150-6-50