Training for ultra marathons has led me to hours at a time either hiking or running in various temperatures. Again, I'm no professional and I'm sure you could do better somewhere else, but here is my basic understanding.
I hope it helps:
3 Basic Layers
-First Layer (Base layer)
Call me old-fashioned (or cheap), but I'm not into spending lots of money on high tech running/hiking clothes. I always figure, Surely there is something in my wardrobe that will work. But your first layer, I have learned, is very important! It may be worth buying a good sweat-wicking shirt. There are good microfiber shirts that are inexpensive (you definitely don't have to go with the expensive brands, but you will be very happy if the shirt does what it is supposed to do--wick the sweat!) However, if you have any shirts that are primarily polyester, they will do. I often just throw a cotton t shirt on before a run, but I assure you that is a big mistake! When I do, I feel every bit of sweat, and when it is below freezing, the sweat freezes and leaves you very uncomfortable.
-Second Layer (Mid layer or Insulating layer)
In temperatures 40 degrees or above, you might be okay just grabbing a sweatshirt, but if it gets down to 30 or below, you are going to want to think a little harder about how you insulate. Of course, there are many different types of insulation (cotton, down, wool, fleece...). To me, I would rather go with two lighter weight shirts for a mid layer than a thick, heavy coat. Also, remember that you can insulate too much. If you are over sweating, this layer is going to get soaked and it will freeze.
-Third Layer (Shell layer)
This layer should not be very thick at all. It is not for the purpose of keeping you warm (that is the point of the second layer). This layer is to keep the elements from getting to you. It should be breathable. Some water proof jackets have vents under the armpits or in the back. Mostly, with this layer, you are just wanting it to be water proof and wind proof. If it is not windy or raining, you probably don't want it at all, but take it with you (pack it away or tie it around your waist) if there is any chance of precipitation, or the if the wind might pick up.
When To Use Hats, Gloves, and Scarfs
In temperatures in the 30's or below, you definitely want a hat, but I would start with a fleece or wool headband. The picture above-left shows me before a run in 20-30 degree weather. I often wear this fleece headband. It covers my ears and as much or as little of my head as I want. If your mid layer has a hood, you can put it on when you are feeling cold, and flip it off when you need to "let off some steam."
In the picture above-right, it is 5 below and I have a fleece headband that I can pull down to use as a scarf, a fleece hat that barely covers my ears (big head!), and a hood from my mid layer that I can pull on and off as needed.
Another benefit I have found with this type of glove is that I can keep my cellphone inside and it will stay warm. My battery will read "dead" if it is below freezing, but when warmed up, I can get in a quick phone call if needed. I'm not sure about newer phones...I still have an old, small, flip phone!
I hope this helps. If you have any other questions, I would be glad to answer them the best I can. But for now, I feel like this post has reached my limit in length. Thanks for reading. Stay warm!