Before I go any further: for a multitude of reasons, you should use an
appropriate method for accessing SQL Server. There are loads of options:
ODBC, JDBC, ADO, ADO.Net, DB-Library, etc, etc. That being said...
Ignoring SQL Server security for the moment, the data sent to/from SQL
Server is in a format known as TDS that is proprietary and (AFAIK)
undocumented. There is an open source project known as FreeTDS you can
check out (freetds.org) if you're so inclined.
The above goes for talking to SQL Server over any protocol, including TCP
(usually port 1433) and Named Pipes. The only "easy" way I know of for
"talking" to a SQL Server via "raw" code is UDP port 1434. If a SQL Server
with a default install gets "\02" on this port it will send back a message
with various fields you can read (TCP port, Named Pipe name, etc). However,
I don't know how much this will help you (especially since many people are
blocking this port for security reasons).
Bottom line: if you want to execute SQL on MSSQL, use technology designed
for it (see my first comment above).
The only option that I know of (and I really, really *don't* recommend this)
is to create a gateway service that listens for a request, submits it to SQL
Server via ODBC or OLEDB, and then returns the results to the original
request. However, what you've essentially done is to bypass SQL Server's
security mechanisms. If this gateway service connected to SQL Server with
admin credentials, you can forget worrying about small time stuff like
buffer overflows: a malicious user could create a Windows admin account in
about 2 seconds.
Craig