Tech Support > Computers & Technology > Programming > sending HTTP requests... newbie (continued)
sending HTTP requests... newbie (continued)
Posted by TreatmentPlant on October 21st, 2007


hi again...

I have written my code that connect to the server (on port 80 - I don't
know if I should be trying something different?) but the "request" is
not being actioned by the server-side app.

I would be most appreciative if someone could point me in the right
direction to get the "sendBuff" message to the server and read back the
response into the "recvBuff"

When I run this code, the program just hangs awaiting the repsonse,
which never arrives so I am assuming the message I am sending is not
getting handled correctly? (Probably because I am not sending it properly?)

Are there any other issues that I should look at?

TIA...

P.S. Windows environment, ANSI C where possible...




//client.cpp
#include <winsock.h>
#include <iostream>

int main()
{
SOCKADDR_IN server;
SOCKET sock;
WSADATA wsadata;
char sendBuff[] =
"http://www.theserver.com/http-api.php?action=send&user=USER&password=PASSWORD&fr om=FROM&to=TO&text=TEXT%20TEXT%20TEXT";
char recvBuff[4096];

if(WSAStartup(MAKEWORD(1, 1), &wsadata))
{
std::cout << "WSAStartup error.\n";
return 0;
}

server.sin_family = AF_INET;
server.sin_port = htons(80); //the port
server.sin_addr.s_addr = inet_addr("127.0.0.1"); //the server

sock = socket(AF_INET, SOCK_STREAM, IPPROTO_TCP);

if (sock == SOCKET_ERROR)
{
std::cout << "Socket error.\n";
return 0;
}

if (connect(sock, (SOCKADDR *)&server, sizeof(server)) == SOCKET_ERROR)
{
std::cout << "connect error.\n";
return 0;
}

send(sock, sendBuff, strlen(sendBuff), 0);

while(1)
{
if (recv(sock, recvBuff, strlen(recvBuff), 0) == SOCKET_ERROR)
{
std::cout << "recv() error.\n";
return 0;
}
std::cout << recvBuff;
};

return 0;
}

Posted by moi on October 21st, 2007


On Sun, 21 Oct 2007 22:45:49 +0800, TreatmentPlant wrote:



What is recvBuff ?

What is strlen ( recvBuff ) supposed to do ?

HTH,
AvK

Posted by Logan Shaw on October 21st, 2007


TreatmentPlant wrote:

You're talking to an HTTP server, but you're not speaking its language!
You have to speak HTTP to the server, not just send the URL you want.

HTTP works by creating requests and responses. You probably want to
create a request that contains a "GET" method and then read (and parse)
the response that comes back.

You should look at sections 5 and 6 especially of the RFC:

http://www.w3.org/Protocols/rfc2616/rfc2616.html

Basically, you want to send a "GET" line (actually a method) followed by a
"Host" line (a header that goes with the method). By the way, a common error
when sending HTTP requests is using some line terminator other than CR LF.

To answer your question about port numbers, you should parse the URL to
get the port number. 80 is the default for http, so you use that if you
don't find one. Your code is also connecting to 127.0.0.1, and I assume
that is just an example, but you should get the hostname or IP address
out of the URL as well. (And if it is a hostname, you should resolve it
into an IP address.) Of course you will need to have parsed out the local
part of the URL in order to send it in the GET method. (The GET should
not include the "http" part or the hostname part of your URL.)

Finally, if you are doing this for the fun of learning HTTP, then great.
However, if you don't care about learning HTTP and merely want to transfer
some data, you should probably abandon the idea of implementing HTTP yourself
and use a library. Doing it yourself is a lot of work, and it's easy to get
it wrong. There are free libraries, like libcurl, available.

- Logan

Posted by Phlip on October 21st, 2007


TreatmentPlant wrote:

El Goog will lead you to any of dozens of libraries that do HTTP for you,
without you reinventing the wheel.

If you just want to dump a web page, I would use something like:

popen("lynx -source my.site.com");

Why C? You are not writing a device driver, are you?

If you just pick a softer language, it would come with a bundled HTTP
package.

--
Phlip



Posted by Aki Tuomi on October 21st, 2007


Phlip kirjoitti:
And there are lots of good ones for C as well, like cURL.

Aki

Posted by Richard Heathfield on October 21st, 2007


Phlip said:

<snip>

That's up to him, isn't it?

C is not just for writing device drivers. It is for writing things that
people choose to use C to write things in. This group is supposed to be
language-blind. Let's leave language wars at the door, shall we?

--
Richard Heathfield <http://www.cpax.org.uk>
Email: -http://www. +rjh@
Google users: <http://www.cpax.org.uk/prg/writings/googly.php>
"Usenet is a strange place" - dmr 29 July 1999

Posted by TreatmentPlant on October 22nd, 2007


Logan Shaw wrote:

Ok... I have decided that I shall try again to get curl installed for
use... I use MS Visual Studio 6, i.e. MS Visual C++ so all the
instructions, make files etc are not working for me...

I have downloaded : libcurl-7.15.5-win32-msvc
The file contain some "installation" instructions, but these appear to
be out of date; files mentioned don't exist etc

Where can I find a simple set of instructions on how to do this in a
windows environment? I have spent 5 hours this morning downloading
different versions of curl, tried following the (in some cases pathetic
or obfuscated) installation instructions contained therein, with no success.

Can someone please provide assistance on how to get the relevant files
into the relevant places so that I can create a simple network app?

I can copy the header files to the appropriate include directory, but I
cannot build the implementations .lib file, I have a .dll file that
appears to be left over and it doesn't matter where I try to copy this
to, any program using curl cannot seem to find it?

In simple terms, please help...


Posted by Phlip on October 22nd, 2007


TreatmentPlant wrote:

Get the command-line version of curl and shell to it.

Or, if this libcurl comes with a compiled LIB file, add it to your project
library list, and add the path to its .H file to your header folder list.
Start at something like Project Settings.

Why are you using C for a program that should be a simple script?

--
Phlip



Posted by Logan Shaw on October 22nd, 2007


TreatmentPlant wrote:
I downloaded a file that I think is the same as that and looked around,
and if I had to guess, it looks like it is probably a library that was
built with the gcc compiler (or maybe with cygwin) on Windows.

If I were trying to get this working on Windows (which would be a challenge
for me since I've never done C programming on Windows), I'd download the
source instead and use my IDE thingy to make a project for it from scratch.
That way I'd hope to get a binary that would work with my compiler. (I
don't think gcc and MS Visual Studio work similarly, although I don't
really know.)

- Logan

Posted by Richard Heathfield on October 22nd, 2007


Phlip said:

<snip>

Perhaps he likes C. Why are you criticising other folks' language choices?

--
Richard Heathfield <http://www.cpax.org.uk>
Email: -http://www. +rjh@
Google users: <http://www.cpax.org.uk/prg/writings/googly.php>
"Usenet is a strange place" - dmr 29 July 1999

Posted by TreatmentPlant on October 22nd, 2007


<snip>

After many more hours of fiddling to get the right files into the right
places, find the missing .dll files, rediscovering that a lot of file
are missing from the different versions of the download libraries,
rebuilding .lib files etc.... I finally have a "working product" (of sorts)

My next problem...

I can send my data to the server and I get the response back, but the
response comes back as a CURLcode datatype. I want to do some more
processing on the server repsonse, so am wondering how can I get the
response cast(?) as a char array or string? or copied into another
datatype variable?

Reading through the curl help files they mumble something about writing
the results to a FILE* but I just need to so some text processing,
parsing, tokenising etc on the results, I don't want to have to write to
a file, then read from the file as I think this double handling
unnecessary. If the results are not written to file, it is sent to
stdout by default. Maybe there is some way to catch the stdout into a
variable (stdout can be redirected to file, can it be redirected into a
variable?)


The code I am using is below, with obvious privacies hidden...

TIA


P.S. I thank people for the time they spend reading my posts; if I
wanted to do this in *nix or as a command line script I would, but I am
trying to improve my programming skills (with a lot of difficulties, and
very little luck) There are some awesome programmers out there who have
some awesome knowledge, please keep your responses within my scope and
ability.



#include <curl/curl.h>

int main(int argc, char *argv[])
{
CURL *curl;
CURLcode res;

curl = curl_easy_init();
if(curl)
{
curl_easy_setopt(curl, CURLOPT_URL, "http://www.theserver.com/api.cgi");
curl_easy_setopt(curl, CURLOPT_POSTFIELDS,
"action=send&user=USER&password=PASSWORD&from=FROM &to=TO&text=TEXT");

res = curl_easy_perform(curl);

// always cleanup
curl_easy_cleanup(curl);
}
return 0;
}

Posted by Phlip on October 22nd, 2007


TreatmentPlant wrote:

Write the source to a temporary file, and read it back as a string. This is
just as fast, on a virtual memory system, as using a library that writes the
string directly. You are talking about wasting programmer time - researching
how to use Curl outside its comfort zone - under the assumption this will
save program time. Don't prematurely optimize.

Next, I would use Tidy to convert that file to XHTML, as
tidy -i -asxhtml -m, and then I would use an XML parser to do these
manipulations. These two systems will handle all the encoding and markup
issues you can think of, automatically.

Again, a softer language would come with many of these tools already
available for you...

--
Phlip



Posted by Logan Shaw on October 23rd, 2007


TreatmentPlant wrote:
It looks like from the documentation that receiving the data can be
handled by a callback function. That is, you want to define your own
function, then after curl_easy_init() but before curl_easy_perform(),
you want to call curl_easy_setopt() with the CURLOPT_WRITEFUNCTION
constant and the pointer to your callback function. Then when you call
curl_easy_perform(), that will cause your callback function to get run
however many times is necessary to get all the data transferred to you.
You probably also want to call curl_easy_setopt() with the CURLOPT_WRITEDATA
to define the void *stream value that will be passed to your callback.

Basically, libcurl is very flexible and lets you do whatever you want to
with the data, and it does this by providing an interface where it can
call a function and say "here's the next piece of data".

Here's some partial code for an implementation that might write it to
an array and resize that as necessary:

typedef struct CurlBuffer
{
size_t allocated;
size_t used;
char* contents;
} CurlBuffer;

size_t curlReallocWriterCallback(
void *ptr, size_t size, size_t nmemb, void *stream)
{
CurlBuffer *buffer = (CurlBuffer) stream;
size_t bytesIncoming = size * nmemb;

size_t bufferSizeNeeded = buffer->used + bytesIncoming;
if (bufferSizeNeeded > buffer->allocated)
{
void* newContents = realloc(buffer->contents, bufferSizeNeeded);
if (newContents == NULL) { return 0; }

buffer->contents = newContents;
buffer->allocated = bufferSizeNeeded;
}

memmove(buffer->contents + buffer->used, ptr, bytesIncoming);
buffer->used += bytesIncoming;
}

void foo()
{
CURL* handle = curl_easy_init();
CurlBuffer buffer;

buffer.allocated = 0;
buffer.used = 0;
buffer.contents = NULL;

curl_easy_setopt(handle, CURLOPT_WRITEFUNCTION, curlReallocWriterCallback);
curl_easy_setopt(handle, CURLOPT_WRITEDATA, & buffer);

curl_easy_setopt(...... whatever else ......);

CURLcode curlCode = curl_easy_perform_handle();

curl_easy_cleanup(handle);
free(buffer.contents);
}

This code won't be efficient, and the error-handling is omitted, but
hopefully it gets across the general idea. You could also, instead of
buffering everything up front, parse it as it comes across in the stream.
That is, you could make the callback function call your parsing code.

- Logan

Posted by toby on October 23rd, 2007


On Oct 22, 7:21 am, TreatmentPlant <NoEmailTha...@DieSpammers.com>
wrote:
You know, I could take the high ground... but... "Welcome to Windows."


Posted by toby on October 23rd, 2007


On Oct 21, 5:34 pm, Richard Heathfield <r...@see.sig.invalid> wrote:
Let's not leave common sense at the door. Reinventing the wheel is
pointless if not for autodidactic purposes.



Posted by Richard Heathfield on October 23rd, 2007


toby said:
Agreed, hence the recommendation (by someone or other) of libcurl, which I
warmly applaud. And if he wants to access it from C, good luck to him,
says I.

--
Richard Heathfield <http://www.cpax.org.uk>
Email: -http://www. +rjh@
Google users: <http://www.cpax.org.uk/prg/writings/googly.php>
"Usenet is a strange place" - dmr 29 July 1999

Posted by Phlip on October 23rd, 2007


toby wrote:

toby, Richard. Richard, this is toby.

He snipped my conclusion to complain about my premise:

"Why C? You are not writing a device driver, are you?

"If you just pick a softer language, it would come with a bundled HTTP
package."

I have lost count of the number of times I have whipped out some soft
language (Perl, Python, Ruby, etc.), downloaded a web page, parsed it, and
went on with my project. Without asking low-level questions to an off-topic
newsgroup.

--
Phlip



Posted by TreatmentPlant on October 24th, 2007


<snip>

Thanks for your help Logan. Your solution did not completely solve the
problem, but was the inspiration and foundation needed for me to get the
thing to work!

Sensational work. Thank you again.


Similar Posts