When you first go buying supplies, take a look in your Sunday newspaper fliers and/or the internet web pages for your local craft store. Around here (northern NJ) it would be ACMoore, Michaels, Joanns. All 3 have web coupons for 40-50% off one item, and/or 20-25% off the total order (sometimes with exceptions) but still, good deals. ACMoore and Michaels have Sunday fliers in the Star Ledger.
I'm fairly certain that each retailer will cross-honor the others' coupons, but with exceptions. Do use a 50% or 40% off coupon for a set of hooks or knitting needles (of course, you'll want to use it on your most expensive item, whatever it is).
When you are web surfing before you shop, pay attention also to what yarn is on sale. When you're new to crochet or knitting, you're going to want to start with worsted weight yarn. Buy what's on sale, because it really won't matter, except to your budget. Personally, I'd suggest Red Heart Super Saver in a light color (I do not suggest Caron's Simply Soft for learning), a size I crochet hook, or size 8 knitting needles. That right there is all you need to begin.
Eventually, you'll want to add supplies to your craft kit, like small sharp scissors, a fabric tape measure, darning needles. Stitch markers can purchased, or use paper clips, bobby pins, safety pins. Even a scrap piece of contrast color yarn will come in handy as stitch marker and, in fact, many people when crocheting amigurumi prefer the piece of yarn because rather than removing and replacing it, it can simply be left to run up alongside of the project until its done then .... just pull it out. Row counters are handy for knitting as are gauge rulers.
It doesn't hurt to have extra hooks/needles handy. Oftentimes we work on several projects at a time. It doesn't so much matter to crochet--you'd pull up a long loop to not lose your place. However, with knitting, necessarily your needles will be left behind with the project holding all the stitches in place.
Don't believe me? Think you'll never have more than one project going at a time? Trust me, you will.
Look at Ebay for oddlots of hooks, needles and accessories. Before you bid....know that you're getting a deal. Know ahead of time what that lot would cost you if purchased new. You'd wind up kicking yourself if you paid too much for used items...but speaking from experience, I've won some lovely craft lots on Ebay.
Also, if you are the garage sale type, you can sometimes find hooks/needles/etc.....but beware of second hand yarn. If it looks old, it is. If it looks raggedy, it is. If you have it in you to "sniff" garage sale yarn, take a small wiff. If it smells musty or smoky, its better to leave it behind. You could purchase it and wash it intact (stuffed into the legs of old pantyhose), but I've heard horror stories about garage sale yarn. Sometimes, mice had set up house in it and you won't know until you got to the middle and found some chewed ends and droppings. Ewwwwwwwwwwww. Beware of garage sale yarn. Also, if someone gives you yarn, examine it carefully and use your common sense whether or not its usable or should be trashed.
When you've learned how to crochet or knit, and you decide that you want to continue doing either (or both), you have plenty of time to move on to fancier more expensive yarns and treat yourself to pricey implements, gadgets and accessories, if you want to.
It doesn't have to be expensive when you start -- no one is happy with a first project cashmere potholder~!