Sure, you can do your best to use only native Japanese words but don't be surprised if you are looked at strangely.
Imagine if you tried to only use "native" English words when speaking with your friends:
1. Do you guys want to go to "singing by yourself" box so we can sing?
2. I love eating "raw fish" at the "raw fish" restaurant near my house.
I do a lot of interpreting work and I honestly do not like saying any katakana words. But even when I know the Japanese word (when they exist) I am many times told that the katakana version is preferred because it's what they are used to hearing.
I have even had some of my interpreting clients ask me what the word I was saying meant. This happens when I am using words that an older engineer has taught me and I then use that word with a younger engineer.
So... there is no way to avoid them. The good news is you can easily remember these words.