I think I would try again! I also think the darker patches are where it has not been sanded!
I would sand it all back using 180 g, then add more grain filler, then sand again with 180 g, then stain. (Most forum members suggest that you should not go above 180 g if you are staining (as it tends to polish the wood, which then does not absorb the stain evenly).). You may then find that the stain take up is darker (like on the edges and the rough spot). If you do not want it the darker colour you may need to dilute your stain.
I also agree with Drashkum that it looks like the sanding was not all "with the grain" which has left some minor scratches. I tend to use a sanding block to help keep the sanding flat.