The news

http://www.acm.org/technews/articles...6f.html#item10

says that a 1000-processor system has been made to simulate
the brain. It claims to realize 20 billion neurons and 20
trillion connections. Given that the brain is estimated to have
about 100 billion neurons (i.e. only 5 times more), this sounds
certainly very impressive. I have a presumably very dumb question
related to programming: How could one run an extremely huge
network of 20 trillion connections 'practically' (i.e. with good
efficiency) on a system having only 1000 processors? Intuitively
that seems to me to be very hard, especially since the natural
neurons to be simulated commonly may have rather distant, i.e.
non-local connections. (I didn't ask the company for an answer,
since I suppose that most firms are reluctant to provide such
technical details for some commercial reasons.)

Thanks.

M. K. Shen

Timesharing (multi-processing): If an emulated neuron needs to make a
decision only once ever hundredth of a second, and the CPU can make 500
million primitive arithmetic/logic calculations per second, and it
takes only 50 such primitive operations to emulate one neuron decision,
then it takes only a ten-millionth of a second to emulate one neuron
decision, after which the CPU can switch to working on another neuron
decision during the next ten-millionth of a second, etc. completing a
hundred thousand neuron decisions before it's time to go back to the
first neuron and start working on the next decision for it. So each
single CPU could emulate 100,000 neurons, and with 1000 such CPUs, a
total of 10 million neurons could be emulated. With faster CPUs, and/or
fewer primitive calculations per neuron decision, more neurons can be
emulated. But 20 billion neurons seems a bit beyond current technology,
but then I don't know how fast current techology is, so I don't know
whether the claim is reasonable or not.

A hundredth of a second is plenty of time for a speed-of-light signal
to travel from one CPU to any other CPU in the same building, so that
the output from one iteration can be fed to input for the next.

However nobody has a decent model as to how a human brain is connected,
so I doubt the simulation could be correct in the first place.

Mok-Kong Shen <mok-kong.shen@t-online.de> wrote in news:cdqmpe$qtl$04$1
@news.t-online.com:

<snip>

That's easilly dealt with: you virtualise connections in such a way that
you can 'move' a neuron from being simulated by one machine to another,
and then arrange it so that a whole locallity happens to be on the same
machine. The next step is optimising the interconnections which do cross
computers, but by reducing their number you've already achieved greater
communication efficiency anyway.

If course, the one thing they don't mention in the artical is how well
'neurons' and 'connections' are simulated. It might very well be no
simulation of reallity at all, and merely a model which someone once
thought might work a little bit like real wetware (i.e. most things
called a neural net).

Ian Woods

--
There is no sig.

RobertMaas@YahooGroups.Com wrote:

[snip]
There could be other issues than that of speed. The complex
(presumably also rather irregular) 'topology' of the network
needs to be adequately represented and managed for performing
reasonable computations with it, I would think.

M. K. Shen

Before any simulation can be contemplated, I presume somebody has a
mathematical model detailing the logic of each neuron, the connections
between them, and the timing of events whereby one neuron must have
finished generating its next output so that the next neuron(s) that use
that output as their input can see it correctly. The topology is simple
via pointers. For fast processors simulating many slow neurons, the
scheduling of timed events is simple via a priority queue: The
computation whose output will be needed soonest, gets run first, then
whatever's output will be needed next is run next, etc. A heap (as in
heapsort) in each processor should suffice. The only non-trivial thing
would be load balancing.

Ian Woods <newspub2@wuggyNOCAPS.org> coughed up the following:
To some extent it almost doesn't matter.

The thing is "thinking", as does a swirl of leaves and my toaster oven. The
question is /how well/ and /how recognizably/ so.



--
http://www.allexperts.com is a nifty way to get an answer to just about
/anything/.