Skip to main content

Catching Corrupted Photos; and Big Bird Brains

In this episode, animal behavior expert Bernd Heinrich discusses his article in the April issue of Scientific American on animal intelligence, and Adobe Systems scientists David Story and Martin Newell talk about methods in development to enable investigators to spot phonied-up digital images. Plus we'll test your knowledge about some recent science in the news.

Uncertain

Welcome to Science Talk, the weekly podcast of Scientific American for the seven days starting April 11th. I am Steve Mirsky. This week on the podcast—the forensics of photos and bird brains. We'll talk with animal behavior expert, Bernd Heinrich, about raven intelligence and a couple of researchers from Adobe Systems will discuss forensic photography, methods and development to root out image manipulation. Plus, we'll test your knowledge about some recent science in the news.

First up, Bernd Heinrich. He is a professor of biology at the University of Vermont. He has written seven articles for Scientific American, the latest co-authored by Thomas Bugnyar is in the April issue and is called "Just How Smart are Ravens." I called Heinrich at his office in Burlington, Vermont.

Steve: Hi Dr. Heinrich. How are you today?


On supporting science journalism

If you're enjoying this article, consider supporting our award-winning journalism by subscribing. By purchasing a subscription you are helping to ensure the future of impactful stories about the discoveries and ideas shaping our world today.


Heinrich: Fine thank you, Steve.

Steve: Tell me, first of all, all ravens are crows, but not all crows are ravens. Is that right?

Heinrich: Yeah! I mean ravens are a select group of

ascrows, generally larger birds and, you know, the raven that we are talking about here is Corvus corax, which is the, the raven that most people think of—it's all over the Northern Hemisphere, Asia, Europe, North America down to South America; so it's huge distribution.

Steve: But, for example, in the New York metropolitan area, if I see a large, black bird on a telephone pole, is that a raven?

Heinrich: No that's a crow.

Steve: That's a crow—okay. So and a raven would be noticeably larger than a crow?

Heinrich: Noticeably yes.

Steve: Okay, good. So now the article is called, "Just How Smart are Ravens." So how smart are they?

Heinrich: Well! You know, it's hard to say compared to what.

Steve: And what do we mean by "smart" when we're talking about another species?

Heinrich: Exactly! So, you know, different animals have different intelligences, they can do different things, and, you know, I think what we were concerned about mostly is whether or not they can reason and whether or not they distinguish other individuals and make distinctions between other individuals on the basis of what they might know or how they might react to them. There are many different things, I mean, one can look at tool use or any number of things for intelligence, but I think mostly what we are talking about is our responses that are rather complex, that are not strictly programmed by evolution, that are hardwired. For example, some insects can use tools even and do some very complex things like build nests which are, you know, very complex, but we wouldn't call it intelligence because it's not really smart, they are programmed that way. And on the other hand we want also to distinguish between responses that are learned by trial and error. You know, you can teach a snail or a bee to distinguish and learn in a few trials but then you can shape the behavior by having, you know, one response after another added on, like the psychologist train[s] rats to do all kinds of fancy things by rewards and punishment one step at a time, but you don't have to know a thing about why they're doing it.

Steve: Why don't you talk a little bit about one of the experiments that you go into detail about in the article, where the birds have to pull up the string to get the food?

Heinrich: Yeah! Well! I started that string experiment kind of just by happenstance. Somewhere back in the early 90s I had a bunch of ravens for various other reasons and these are tamed birds that I had at the aviary at the house and, you know, I knew what their background was. They've never seen strings, never had food presented in any such way, and I knew that in the wild, ravens are scavengers who feed mostly on dead animals with carnivores, the wolves and so on, and provides, so they don't get food, you know, that's suspended on something long, so they can't be genetically programmed for that for getting food that's suspended on a string that's about a meter long.

Steve: Right! this is a novel situation.

Heinrich: A novel situation, right. And I had never before, you know, presented food to them that way, so it's just a very cheap way, just curious, so I stuck, you know, a string out there and put food on there and expected that they would maybe fly at it and try to grab it; do the obvious simplest thing which wouldn't get them any food because you can't swallow it dangling down. To my great surprise, after awhile it went over there and kind of looked at the situation and pecked on the string a little bit, and it reached down, grabbed the string and pulled it up and then set it onto the perch, put a foot on it, then looked around, reached down pulled up another loop and stepped on it, etcetera, stepped on that loop etcetera and until it got the meat, which was hard salami and so, you know, I was totally blown away because that seemed to suggest that the bird had an understanding of what it was doing; and so I had to get birds interested one at a time and I had to get them used to string by having it available but not dangling, I mean, I just had it on the wall untied so that they could, because I didn't want them to be afraid of the situation, not to be afraid of strings. Ravens are extremely touchy, they are afraid of anything that's new.

Steve: But interestingly they are not apparently afraid of sneaking up behind a wolf and pecking at its backside.

Heinrich: Exactly! (laughs) That is a fascinating thing about them. You know, this is a play behavior that has evolved for them to get acquainted with carnivores that normally provide them with food.

Steve: Let's go back to the string experiment just for one more moment and follow up on how it really indicates a kind of an intelligence rather than an instinct—this behavior—because it actually appears that there was some serious preparation involved. They actually seemed to be studying the situation and only taking action after having formulated some kind of a plan.

Heinrich: Yeah! Well! I don't know if they had actually formulated a plan. All I know is that they hesitate a lot, which might be due to that they formulate a plan; on the other hand it might be just that they are afraid. But the point was that once they got there, went on the string, they were not afraid of it, and then some of them could do it very quickly and others not. First of all the ones that couldn't do it were the young birds. They would solve the problem in about six minutes after they started, but they would test all kinds of things. They would pull on the string, drop it, peck it, twist it, jump at it and all kinds of stuff like that—all different things—and then they would do it. The adult birds would bypass a lot of those, what to us seems obvious solutions, which don't work, as though they had tried them in their mind, the same way as we do.

Steve: This is a general kind of question about the theoretical difficulties in working in this field: When you are dealing with another species how do you continually try to make sure that you are leaving your assumptions about what intelligence is at the door and really objectively studying intelligence?

Heinrich: Well! I know objectively that they did not have the experience before. I know objectively that they could not be programmed to do the whole sequence. I mean they might be programmed—genetically have programming—to peck and pull, but to go through a whole sequence as of applying that to the situation is not something that the whole sequence could be genetically programmed, because that's not how they find or get food in the wild.

Steve: Right! But, in general, when you are designing an experiment you must have to keep that at the forefront of your mind at all times. Is that right?

Heinrich: Exactly! Exactly! You got to have the natural history of the animal in mind and you have to know a lot about the animal's lifestyle and you have to know the animals first of all.

Steve: Or else you might wind up not testing the thing that you think you're actually testing?

Heinrich: That's right! But the test in this casemdash;the test is so unique that even if one solves it, because there are so many steps involved we have to do them all in a specific sequence and if, you know, how to do it, then you can pull together a sequence of a couple of dozen steps, and if you don't know it, then each of them has to be programmed specifically in a reward program where, you know, you reward one and then you add on another and another and another ....

Steve: And that should take weeks?

Heinrich: That should take weeks and weeks. Yeah!

Steve: So the article again is called "Just How Smart are Ravens." It's in our April issue of Scientific American and there is a lot in there about the interactions between ravens and wolves and specifically the interactions among ravens with each other and how interesting those are because this is an amazingly social animal that has complex relationships with its kind-specifics there.

Heinrich: Yeah! This is one of the things that possibly makes them more intelligent than other birds is their real social behavior. They have to know each other, they are confronted with dangerous carnivores at the food and have to interact with each other and predict each other's behaviors, and so all of this kind of came out, you know, through this study that was going from one little step to the next—food finding behavior, how did it determine what's good to eat and how they interact with all of these things, which are relevant in the environment.

Steve: You have a book coming out in the next few weeks I understand.

Heinrich: Yes it's called "The Snoring Bird" and it's a lot about my father—actually he was one of those Victorian-type explorers who went all over the world collecting birds for different museums, and one of the birds that was very much desired by Leonard Sanford at the American Museum of Natural History in 1930s was a rail that was thought to be extinct from Indonesia and my father had contacts with Erwin Stresemann who is one of the premier ornithologist[s] in the world at the time. He and Sanford were friends and they met in Berlin and my father was there. He had gone to—he had collected some birds in Bulgaria and written a little paper on it and shown it to Stresemann, and they decided that he might be a good collector to go after it, and so he and three women (laughs) went out to Indonesia, and he spent three years—no two years—looking for the bird, and near the end he found it and that kind of earned his stripes as an ornithologist collector and ultimately that brought us to America, and so, you know, I kind of talk about the legacy of [an] early Victorian naturalist and my father in particular and his adventurous life and how my relationship to him and ....

Steve: And the book again is called "The Snoring Bird."

Heinrich: "The Snoring Bird." Yes it refers to a rail of Indonesia which its vocalization sounded like a snore it turned out at the end.

Steve: Aha....

Heinrich: And that's how he tracked it down.

Steve: So this is sort of a microcosm of that whole era of Victorian exploration and collection?

Heinrich: And in part it's autobiographical, too, because I relate how my evolution as a biologist stemmed from my father.

Steve: Dr. Heinrich, thanks very much. I appreciate it.

Heinrich: Thank you very much Steve.

Steve: "Just How Smart are Ravens" by Bernd Heinrich and Thomas Bugnyar is in the April issue of Scientific American. It's also available at our Web site, www.sciam.com.

Now it's time to play TOTALL.......Y BOGUS. Here are four science stories. Only three are true. See if you know which story is TOTALL.......Y BOGUS.

Story number 1: Cranking up the Large Hadron Collider in Europe will be delayed because of a math error in the construction.

Story number 2: Genetic analysis shows that the famously feuding families, the —Hatfields and McCoys—are actually very closely related.

Story number 3: A new study finds that about half of the world's species of magnolias appear to be in jeopardy.

And Story number 4: Good news about chocolate: cocoa appears to have a beneficial effect on blood pressure.

We'll be back with the answer, but first I recently sat down with a couple of bigwigs from Adobe Systems, whose products you probably use a lot—from Acrobat to Photoshop.

David Story is Adobe's vice president for digital imaging engineering. Martin Newell carries the title of Adobe Fellow for four decades. He has been a computer scientist and software engineer. They were visiting New York from their home base in San Jose, California, and we met to talk about the future of digital imaging. We talked for a good 90 minutes. I'm saving some of the discussion for a future episode or two, but I'll tell you this, it's going to soon become almost impossible to take a bad picture.

This week, though, I want to play you the part of our conversation that dealt with forensics. Image manipulation is becoming so sophisticated. How do you make sure that an image in the news or in court hasn't been tampered with? By the way, we were in a boardroom with a big window facing Broadway and with lots of air coming out of ducts, so that's the extra bonus sounds that you're going to hear. David Story has the American accent, Martin Newell has the British accent.

Steve: Are you embedding any information so that for legal reasons you can tell if a photograph has been tampered with?

Newell: Actually, yes. I mean one of the things that's very kind of—I wish it was better known feature of Photoshop—is that there is a thing called the history log. And what it does is, it's the first thing when you open up your preferences; there is check box and it says "record everything I do to this image." And it's a readable format and we've actually worked with law enforcement agencies to make it useful to them. But it's useful to anyone whether they are, you know, science or nature, you know, journals, whether they are receiving imagery about stem cell research, right—whether the, famous, you know, South Korean case where the guy actually cloned parts of the image to make it look like there are more stem cells.

Steve: Right! Cloned parts of the image, not the cells (laughs).

Newell: Right! Right! He was very successful with Photoshop or whatever he used. He just missed a clone step and unfortunately

andnot the other.

Steve: So there could be rules of the evidence whereby the history log must be available on a photograph before it would be admissible in court.

Newell: Yes, and actually what we recommend to people is, we recommend that they capture the original image and have that as backup and submit

thatthe final image with the history log embedded right, and then you can look at it and say, okay that was the original. I can tell what they did grossly. But then I can look at the history log and see if they used any kind of, you know, off-limits tools. With power comes responsibility and if we're going to give people these tools this powerful, we need to help people understand how to detect when something has been tampered. The people who are really good with Photoshop probably aren't getting caught and so this part of our research is actually something where it's based on our ethics. Or is it, couldn't we find some research and support it and advance it that actually will help you completely detect what's wrong with the image? And we actually found out a professor at Dartmouth—Hany Farid—who had been working on this. He had just started to get some results and we said, all right. We jumped in. We helped fund his lab. We put engineers on it, and we put our tech transfer engineer in place. We did student exchanges and so we've come up with some tools now; and what criminologists would like is they like what they have on C.S.I.

Steve: Right! You want to make things that can do what the things on C.S.I. already do?

Newell: Right!

Steve: That's your inspiration.

Newell: Yeah! (laughs) So, because none of those software exist.

Steve: Right! Right!

Newell: So, but what if we had some tools that were about image forensics, and we said what if we could detect whether the color filter array, the sensor image has been authored. Now with a digital camera it turns out that in a camera, the camera itself doesn't sample red, green and blue at every pixel.

Steve: Right!

Newell: Right! It has a red...

Steve: It has one of each.

Newell: Yeah! Exactly. So how does it get red, green and blue at every pixel?

Steve: I don't know.

Newell: It copies somehow. And if you kind of average the color from this blue pixel to that blue pixel you are going to have a very strong frequency signature. We're going to see every other pixel; the information is related kind of right in between and pretty much every digital camera does this. Wouldn't it be great if I could just raise something on the image or run a slog and stick it in the machine?

Steve: And the information reveals itself.

Newell: And see the blood spatter or whatever. In this case it's the spatter of having altered the image.

Story: Essentially it's two parts of the process. The first one doesn't affect all pixels and tries to determine if any pixel is likely to have been some kind of a weighted average of the pixels around it. The simple one would be equal to a nearest neighbor or another one would be interpolated average of two neighbors. Other one is, there are various different algorithms used to fill in those pixels in the digital camera. Some of them are filtered over much larger area to fill in that color. But then the first part does come up with a map of the entire image or of the region of the image that we're inspecting—in this case of how likely it is that, that is correlated with the pixels around it, in this particular way.

Steve: So you get a probability...

Story: Exactly!

Steve: At the end of your computation of doctoring you don't get a definite but you get a 95 percent probability that this has been altered.

Story: That's correct.

Newell: We've actually shown that it's admissible on our court of law. Professor Farid actually does a number of expert consultations, you know, the tools that you actually have, an element that you may or may not have used is the clone stamp and so what we said is, you know, why don't we build some tools that can scan for cloning and we'll look at this image and say: okay let's look at it automatically, the whole image and see, you know, is there anything really suspicious. Mathematically, we looked at the math and said it is improbably similar to this region over here.

Steve: Right! So it's highlighting the two regions that are too close to each other to be chance.

Story: And so really what this is not like we're going to shift this tomorrow. This is based on our ethics we said we need to find research to give people the responsibility and the ability frankly to start to detect some of the tampering that's going on. There is a lot more tampering going on then we're detecting. We need tools beyond the naked eye and so this is the beginning of a tool built, a set of tools that'll help us to detect different kinds of tampering and it's not perfect, you know, it doesn't work in every circumstance yet. Martin, you know, said the other day that it's kind of like antivirus software.

Steve: Right!

Story: Right! We're going to come up with these tools and then people will find a way to defeat them and then they'll come up with better tools, and it's going to be an arms race with the tamperers and there's going to be kinds of tampering that we can definitely detect. Right! Unless you've really gone out your way to obscure it and so the places that are most threatened by the integrity of photographs right now like referee journals, photojournalism and others, you know, and basically some of the basic crime scenes of, you say the detective took this picture. How do I know that the detective didn't, you know, modify it in Photoshop, right? We can give those people tools to help them to prove what they have or haven't done. And so in combination with features that are already in Photoshop, like the history log that we put into help with that, we also want to be able to help just give basic tools and so it's a start. And that's why we're still in the labs, but we hope at some point we will be able to bring this to people so they can actually detect this kind of tampering.

Steve: More from the Adobe guys on the kinds of photos you'll be taking very soon on an upcoming podcast.

Now it's time to see which story was TOTALL.......Y BOGUS. Let's review the four stories.

Story number 1: Delay in the Large Hadron Collider because of a math error.

Story number 2: Hatfields and McCoys are close relatives.

Story number 3: Worldwide threat to magnolias.

Story number 4: Cocoa appears to lower blood pressure.

Time's up.

Story number 1 is true. The Large Hadron Collider particle accelerator suffered a setback because a magnet built by Fermilab in Illinois failed a high-pressure test. The resulting explosion moved the magnet weighing some 20 tons out of position. Fermilab's director said they had committed a pratfall on the world's stage.

Story number 4 is true. Cocoa can lower blood pressure according to research in the Archives of Internal Medicine. Unfortunately, most cocoa products come with a lot of fat and sugar but leaner cocoa dropped blood pressure as much as some medications in a two-week study. Interestingly tea, green or black, did not have a similar effect. For more, check out the April 9th article on our Web site titled, "Cocoa, not tea, calms blood pressure, study says."

And Story number 3 is true. Our magnolias are in trouble. That's according to a report from Bournemouth University School of Conservation Sciences on the southwest coast of the U.K. The study says that

a131 of the 245 species face extinction, which is particularly troubling because the plants are considered indicators of the health of their forest habitats. Two thirds of magnolia species are native to Asia. Chinese temples house individual magnolias, some of which may be 800 years old.

All of which means that story number 2 about the feuding Hatfields and McCoys actually being very close relatives is TOTALL.......Y BOGUS. What is true is that the McCoys may have started it because it turns out that some members of the McCoy family have an inherited condition that can cause a hair-trigger temper and violent outbursts. The Associated Press reports that some McCoy's have Von Hippel-Lindau disease which affects the adrenal glands, which can cause a serious overproduction of adrenaline and catecholamine leading to rage [that] could explain in part why the Enterprise's Doctor McCoy would always get so mad at Spock according to the Wikipedia entry on the Hatfield-McCoy feud—in a Star Trek episode Bones McCoy claims ancestry with the famous clan.

(Music: "I'm trying to thank you, you pointed-ear half-goblin" from Star Trek episode plays.)

Well! That's it for this edition of the weekly Scientific American podcast. You can write to us at podcast@sciam.com, check out news articles at our Web site www.sciam.com and the daily SciAm podcast 60-Second Science is at the Web site and at iTunes. For Science Talk, the weekly podcast of Scientific American, I'm Steve Mirsky. Thanks for clicking on us.

Steve Mirsky was the winner of a Twist contest in 1962, for which he received three crayons and three pieces of construction paper. It remains his most prestigious award.

More by Steve Mirsky
Catching Corrupted Photos; and Big Bird Brains