Does Knowledge Come Only From Our Sense Experiences?

Personally I don't agree with empiricism's view that all of our knowledge is based on the use of our senses. I agree that a lot of what we know comes from what we've experienced before, but not everything. I think in terms of what we see and experience in everyday life - things like our environment, the weather, the date, the seasons, etc, our knowledge does come from what we've seen before; but for more complex ideas this doesn't really apply. 

There are certain things that we just know without ever having to be told or exposed to. I guess these are like our instincts in a way. I've heard cases where people who have never swam before fall into a body of water and they end up being able to swim to save themselves. Without ever being exposed to water, or the feeling of swimming, they were still able to figure it out when they really needed to. Now the act of swimming is a pretty simple idea - one that even if a person had never actually done before, they have probably still been exposed to in some way. But what they really did - the more complex idea, for which I believe the use of senses does not really apply - was survive. They knew that they had to survive and they did what needed to be done. Even though they didn't know how to swim and were terrified of water, something within them knew that was what needed to be done and just did it - without any outside source or past experience to rely on. There are also instances where babies falls into water and they are able to flip over on their back and float, sort of treading their arms, to keep their heads above water. I've seen videos of babies as young as 4 weeks doing this - it was crazy!! This is another good example because a baby that young would never have experienced this before, or experienced anyone else doing it, and even if they did they wouldn't have the mental capacity to understand and remember it to act it out themselves later. It's something they just somehow know to do because they have to - again for survival. The fact that if your head goes underwater and you can't get it back up you will get very injured or die seems to be a universal truth that we are able to know outside of any sense experiences we may have had. So while parts of empiricism apply to most of what we encounter in our everyday lives, I do not believe the idea that ALL of our knowledge is based on the use of our senses because there are some things that are just too complex to completely rely on what we are able to see, hear, smell, touch, and taste.