Hollis is right. It is important to acknowledge family. I think your hesitancy is understandable. Rejection is harder to take from family and loved ones, but Richard has a point. You may be pleasantly surprised to find you are welcomed. They may have looked for you! (possible)

You really have more information than some people have. You have a name, parents' names, place of birth (state, county), US Navy service (military record), and a death certificate with a place of burial. What you haven't said is whether or not you asked your mother what she knew about your father. She may have known the name of a sibling, aunt/uncle or other relative he may have visited (or who may have visited him). Any bit of information, no matter how small it may seem, could help put the pieces of the family together. She may have some pictures of him that his family would love to see. If nothing else, they would be good for 'sharing' when you meet that first relative. [Believe me, pictures can be a good ice breaker!]
Once you learn more about the family, you can begin to determine which direct or collateral relative you would feel more comfortable attempting to contact.

You asked about starting your research.
In general, start with what you have. Begin with the known, as Richard pointed out. For now, you want to locate living relatives. One way is to find / trace family members that were living at the time of your father's death over 25 years ago.

Resources
Death Records --> Obituary
Locate you father's obituary. An obituary may have appeared in the local papers (or the paper for the county of burial / death). Modern obituaries usually mention next of kin (survived by ... preceded in death by ...).

Locate his parents' death record. Depending upon the area of death, they could also have an obituary. Record the names of listed relatives. Once you have names and locations, you can look for contact information.

Marriage Records --> Married names of female relatives, maiden names of their children.
You may have to research a female relative in order to find possible siblings, cousins, aunts, etc. Marriage records can be used to find maiden names (which can be used to find another generation of cousins).

Census research ---> Family groups
you can also begin to put together your paternal family tree.
Based on the information you gave, he should be listed in the 1930 census as a child with his parents and siblings (if any).

His parents would also be listed in the 1920 census and, perhaps, the 1910 and 1900 census records.

There are many books and online articles on how to begin, including one right here on this site. When you get stuck, just post here for help. There are many that are more than willing to help.

A beginning.