In writing, as in the rest of life, context is everything. Just as you would never wear pajamas to a formal job interview, nor would you ever write a formal paper with casual language, slang, jargon, sloppy grammar, and other similar writing deficiencies. Knowing who you're writing for is as important for a writer as knowing what you're writing and why you're writing in the first place. Are you writing for your professor? Your boss? Your friends? Your grandmother? The kind of writing you'd present to each of these people is likely going to vary quite considerably. For your professor, you will likely have formal research papers with a mix of facts and analysis. For a boss, you might have to write memos or progress reports, where raw data is the most important component. Writing for friends and family members can be considerably less formal. Defining your audience becomes all the more important when you consider that "writing" means more than just things in books, magazines or newspapers. The act of "writing" is a broad concept that encompasses anything that uses the written word to convey an idea or emotion: bumper stickers, film and television scripts, e-mails, text messages, notes you send your friends in class, even wikibook texts. In each of these cases, it's important to be aware who you're writing for just as much as it's important to know why you're writing.
At the same time, when you define your audience, you determine what sort of facts you cover in your writing. Say you're writing a paper for a biology class where your audience is your professor (and any other readers in the field of biology), is it necessary to spend a considerable amount of your paper covering background of relatively basic biological concepts? Not at all, and in fact, your professor is more likely than not to think that you're thinking you're padding the essay with obvious information. Now, if you were writing a textbook for a high school biology class, your audience would now be considerably different, and you would naturally explain many more "basic" concepts in detail.
Similar to the idea of writing as conversation, you don't have to agree with the audience you're writing for. In fact, if writing is meant to be a tool to persuade, then you naturally with be writing for audiences who do not initially see from your point of view. Take for example, political writing. Certainly, there are times when political writing is used to speak to those who already agree with the political position and "rally the base". However, it is sometimes most practical for political writing to attack or persuade the opposition to see why their policies are "wrong".