The following gear is all easily obtainable either through non-heroic 5 mans or quests.  I'm specifically avoiding listing PvP reward gear, as it's almost always strictly better than the list below.  However, if you only PvP casually, or are looking for a place to start or get an upgrade outside of the honor rewards, take a look!

I'd also like to jot down a quick note about gearing philosophy as a PvP mage.  In general, your priorities are Stamina -> +damage -> +crit -> everything else.  Crit is more useful for fire mages than frost mages (who can rely on Shatter), but Stamina is still king.  Your base stats that you want to shoot for are 8k hp and +500 damage.  If you're lower than those numbers on either end, work on that as your top priority.   Resilience is also very important, but as that's only really on PvP reward items, just get it as you get "the good stuff".  You should also try to get at least +3% hit total from talents and items, as that is the hit cap in PvP and you do NOT want your counterspells, novas, and sheeps getting resisted when you can at all help it.

Head

  • Evoker's Helm of Second Sight - Fill in with some +stamina gems, then resocket with +damage when you find something better and use it as a PvE helm.  Easily obtained solo through quests, starting with this one.
  • Hydromancer's Headwrap - A very solid PvP helm with a meta slot, obtainable through a quest where you basically just beat Steamvaults.
  • Mana-Etched Crown - Drops off the last boss of Black Morass.  Another meta gem holder with nice stamina.

Shoulder

  • Mantle of Three Terrors - Look like a warlock and have some solid stats to boot!  Drops off the first boss in Black Morass.
  • Spaulders of Oblivion - Drops off Murmer in Shadow Labyrinth.  Very awesome PvP shoulders if you stick some stamina gems in there.

Neck

Chest

  • Imbued Netherweave Robe - Relatively cheap crafted item with a ton of stamina and some nice damage on the side.  Very easy to acquire, or even make yourself if you're a tailor.
  • Robe of Oblivion - Drops off Murmer in Shadow Labyrinth.  Put in some +stamina gems and you have a VERY solid PvP chest with all the stats you need and none that you don't.
  • Vermillion Robes of the Dominant - Besides looking stylish, these have great PvP stats that will last you a while.  Drops off the last boss of Steamvaults.

Cloak

Feet

  • Shattrath Jumpers - Two +12 stamina gems makes these give even more stamina than the PvP rewards, while giving you some nice +damage to boot.  Awesome PvP boots, and obtainable from a quest chain that starts here.
  • Imbued Netherweave Boots - Easily crafted and very solid for PvP.
  • Sigil-Laced Boots - Similar to the Jumpers, these work great with stamina gems in them.  Drops off the last boss of Arcatraz.

Hands

  • Gloves of Oblivion - Nice stamina, decent +damage, can't ask for much more.  Drops off Bladefist in Shattered Halls.
  • Energis Armwraps - Packs more of a punch than Oblivion, and only has slightly less stamina.  Drops off High Botanist in Botanica.

Bracers

  • Shattrath Wraps - That's about it.  Stick a stamina gem in there and these will work.  The PvP bracers are pretty cheap, though, so I suggest replacing these soon and turning them into PvE bracers.  These are obtainable through a quest chain that starts here.

Belt

  • Sash of Serpentra - Solid PvP belt that drops off the last boss in Steamvault.  This is really the only non-heroic drop belt out there for PvP.
  • Hierophant's Sash - Ok, so this is kinda a PvP reward, but there's not much else for belts out there.  20 Halaa Battle Tokens and a single Halaa Research Token makes it yours.

Legs

  • Devil-Stitched Leggings - With stamina gems, this will net you a HUGE hp increase.  Great PvP pants that drop off Laj in Botanica.
  • Trousers of Oblivion - At least they're not "Khakis of Oblivion".  Regardless, once again Oblivion proves itself to be solid for PvP.  Drops in Sethekk Halls off Talon King Ikiss.
  • Imbued Netherweave Pants - Crafted like the other Imbued Netherweave pieces, these are relatively cheap and simple to get while providing you with solid PvP stats.

Rings

  • Burning Blade Cultist Band - Stamina, damage, and crit.  Lower item level than most, but the stats are just fine.  This is obtained through a quest line, available to Alliance here and Horde here.  See the associated comments for prerequisite information.
  • Ryngo's Band of Ingenuity - Less stamina than I'd normally like, but solid stats all around otherwise.  Drops off Wrath-Scryer Soccothrates in Arcatraz.
  • The Black Pearl - Crafted item with a ton of stamina and some decent +damage to boot.
  • Cenarion Ring of Casting - Great PvP ring with a quest chain that starts here.  You'll have to go through Slave Pens and Underbog.  Rough, I know.

Weapons (Staves)

  • The Bringer of Death - Solid stamina, int, damage, and some pretty hefty crit.  Great PvP staff if you can find it on the AH and have some money to spare.
  • Battle Mage's Baton - You can get this at 65 from a very fast (and fun) group quest line that starts here, and surprisingly it'll last at 70 as a PvP staff with all that stamina.  Not so great on the +damage, but if you can make up for that elsewhere this will be a solid weapon for you.

Weapons (Main hand)

  • Mana Wrath - Chock full of PvP goodies, a great main hand that will last you until Gladiator's easily.  Drops off the last boss of Mechanar.
  • Continuum Blade - Very easy to get with even more stamina (but a touch less damage).  Just hit revered with Caverns of Time (very simple with the 8000 rep quest), toss them 45g or so, and you're good to go!
  • Greatsword of Horrid Dreams - Least amount of stamina of the bunch, but the most +damage and a bit of +hit if you're lacking in that department.  Drops off the last boss of Shadow Labyrinth.  This also looks totally badass, which is clearly a selling point.

Weapons (Off Hand)

  • Star-Heart Lamp - Solid PvP stats and a bit of +hit on the side.  Drops off the second boss in Black Morass.  Great to pick up if you're going for the Continuum Blade!
  • Manual of the Nethermancer - Replace the hit with crit on the lamp and you basically get this.  Drops off the crazy pyromaniac Nethermancer woman in Mechanar.

Wands

  • Nether Core's Control Rod - A simple, well rounded wand with a touch of +hit as icing.  Can't go wrong with it no matter what you're looking for.  Drops off the crazy Dalliah the Doomsayer in Arcatraz.
  • Wand of the Netherwing - Holy PvP, this wand is pure stamina and +damage.  If you don't mind losing a touch of mana, this is as good as it gets for 70 blues.
  • Illidari Rod of Discipline - A green wand that has a socket in it, starting with straight +stamina and +damage, with a blue socket and a bonus of more +damage!  Awesome PvP wand for sure, green or not.