Donald Norman introduces the concept (originated by J. J. Gibson) of affordances, which he defines thus (Norman, 1988, p. 9):
When used in this sense, the term affordance refers to the perceived and actual properties of the thing, primarily those fundamental properties that determine just how the thing could possibly be used (see figures 1.5 and 1.6). A chair affords ("is for") support and, therefore, affords sitting. A chair can also be carried. Glass is for seeing through, and for breaking. Wood is normally used for solidity, opacity, support, or carving. Flat porous, smooth surfaces are for writing on. So wood is also for writing on.
Affordances, then, are really literally all the possible actions that a person could perform on an object. You actually cannot see through wood, so it doesn't afford seeing through. "Affordances provide strong clues to the operations of things," he says, but he does not say the converse: just because a thing strongly suggests a certain action to the user, that doesn't make it an affordance.
Norman was annoyed enough by this misperception to attempt a correction (Norman, 1999, p. 40):
Now consider the traditional computer screen where the user can move the cursor to any location on the screen and click the mouse button at anytime. In this circumstance, designers sometimes will say that when they put an icon, cursor, or other target on the screen, they have added an “affordance” to the system. This is a misuse of the concept. The affordance exists independently of what is visible on the screen. Those displays are not affordances; they are visual feedback that advertise the affordances: they are the perceived affordances. The difference is important because they are independent design concepts: the affordances, the feedback, and the perceived affordances can all be manipulated independently of one another.
and (Norman, 1999, p. 40):
Far too often I hear graphic designers claim that they have added an affordance to the screen design when they have done nothing of the sort. Usually they mean that some graphical depiction suggests to the user that a certain action is possible. This is not affordance, either real or perceived. Honest, it isn’t. It is a symbolic communication, one that works only if it follows a convention understood by the user.
It was probably already too late to change the common usage of the term. I know that I first encountered the term "affordance" being used to describe the little "grips" on the corner of a resizable window, which "afford" grabbing and dragging. Not a valid usage, in Norman's view.
There's not much chance of confusion, though. GUI designers cannot change any of the real affordances of the system, after all. And Norman's complaint seems to take things slightly too literally. It's true that it's merely a cultural convention that a beveled rectangle with some text inside represents a button, but granted we accept that metaphor, it doesn't seem much of a leap to say that the metaphorical button affords pushing in the same way that a physical button would. It's merely a question of whether we're thinking of the physical user in meatspace holding a mouse, or metaphorical user represented by the mouse pointer interacting "directly" with UI elements.
So, is Norman wrong to complain? No, but the takeaway isn't that we need to be more precise with our terms. It's this: we should remember that buttons and other such UI elements are merely cultural conventions, and not all users will interpret them the same way. What counts is what users actually perceive and do.