LinuxQuestions.org
Register a domain and help support LQ
Go Back   LinuxQuestions.org > Forums > Non-*NIX Forums > Programming
User Name
Password
Programming This forum is for all programming questions.
The question does not have to be directly related to Linux and any language is fair game.

Notices

Reply
 
Search this Thread
Old 09-20-2004, 12:28 PM   #1
Dark_Helmet
Senior Member
 
Registered: Jan 2003
Posts: 2,786

Rep: Reputation: 368Reputation: 368Reputation: 368Reputation: 368
Neural network persistence


Perhaps this thread's title is not the best, but I couldn't some up with a snazzy title... such is life. Anyway, to the question:

I'm looking for some suggestions on how to propagate a neural network from one portion of a simulation to the next. I've written a neural network class, and have included it in a tic-tac-toe simulator. I have each player in the pool (each with a unique neural network) play two games against every other player (once as player X and once as player O). When all the games are finished, I sort the scores given to each player, and do some player-pool-tweaking.

* Specifically, the top 2/5 of the players are left unchanged -> advance to next round.

* Those same top 2/5 of the players are copied once, with each copy receiving a random "tweak" to their network (only one value within the network is modified) -> they are mutated

* The bottom 1/5 of the players are completely tossed, and new, purely random networks replace them.

After all of that business, the players slug it out again, and repeat the whole process until a sufficient network is found.

Out of a pool of 100 players, I'm not seeing the scores improve at all. It seems to always hove around the starting score. I'm beginning to wonder whether the "population" is becoming stagnant. One obvious thing to try is "genetic crossover" but I was hoping someone might have another interesting idea to try, or would be able to point out a flaw in my logic.

If anyone thinks it's extremely important, I can post the basics of the scoring and "tweak" methods employed.
 
Old 09-20-2004, 01:52 PM   #2
leonscape
Senior Member
 
Registered: Aug 2003
Location: UK
Distribution: Debian SID / KDE 3.5
Posts: 2,313

Rep: Reputation: 47
Just a question but what happens to 3/5 and 4/5 of the players? Since only the 1/5 are thrown away.
 
Old 09-20-2004, 02:24 PM   #3
Dark_Helmet
Senior Member
 
Registered: Jan 2003
Posts: 2,786

Original Poster
Rep: Reputation: 368Reputation: 368Reputation: 368Reputation: 368
The players that don't score in the top 2/5's are lost from the population. They are dropped completely in favor of the "proven" players and their mutations. The new, random nets are intended to preserve diversity (possible introduction of a new strategy).

I think that only keeping the top 2/5's is the crux of the problem. All of them could be clustered around a local maximum, and the simulation becomes trapped. The chance that one of the new, random nets could jolt the simulation out of that situation is probably very remote.

I'll try to implement a probability-based selection routine (greater score implies greater chance to propagate to the next round) and see what that does for me rahter than just taking the top scorers only.
 
Old 09-20-2004, 03:06 PM   #4
leonscape
Senior Member
 
Registered: Aug 2003
Location: UK
Distribution: Debian SID / KDE 3.5
Posts: 2,313

Rep: Reputation: 47
What you might want to do, is rather than just randomly changing a single variable, is take more copies, each of which have one of the variables altered by a set amount, as well as the original.

This means adjusting the you take through to the next round, but gives you more of a chance of propagating useful traits, and strengthing them. where your adjustments at the moment, with only a single copy could simply remove them.

Once you have a net that consistantly beats ots progeny multiple times, you might have an ultimate winner.

Another point is tic-tac-toe, is a game that can be drawn, and may not allow for improvment.
 
Old 09-20-2004, 03:28 PM   #5
Dark_Helmet
Senior Member
 
Registered: Jan 2003
Posts: 2,786

Original Poster
Rep: Reputation: 368Reputation: 368Reputation: 368Reputation: 368
If I follow you, I tried something similar earlier (I may not have had the patience to see it play out though). I took the top 1/5, passed them through, copied them three times with a different mutation to each copy, and then introduced random nets for the last portion (to fill out the population). I got similar results (a little improvement, but the scores seemed to stabilize quickly - a false convergence). Like I said, maybe I didn't give the simulation enough time to improve. It's something I may look into again.

And yeah, tic-tac-toe might not have been the best possible choice, but I thought it was simple enough to cut my teeth on. I have the simulation set up to trigger a solution when a candidate wins half of the matches played, but I also keep an eye out for when the players tie for every game. In theory, at some point, the "unbeatable" defense will evolve, and that suggests the nets would progress from profoundly stupid to winner to stalemate strategy; where the peak point value does not actually represent the best player.

I do appreciate the help. It's good to have an outside perspective on this stuff.
 
Old 09-20-2004, 04:18 PM   #6
leonscape
Senior Member
 
Registered: Aug 2003
Location: UK
Distribution: Debian SID / KDE 3.5
Posts: 2,313

Rep: Reputation: 47
I have this link from when I was messing with NN, and was looking and evolution rather than training which you might find useful. Neuroevolution Methods.
 
  


Reply


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off


Similar Threads
Thread Thread Starter Forum Replies Last Post
Fedora C2 - Wireless network drivers? Broadcom 802.11g Network Adapter FragTek Linux - Wireless Networking 7 01-22-2010 12:10 AM
Slackware 10 network configuration. Problem with 8139 network card drivers ! Padmakiran Linux - Networking 8 03-27-2007 06:48 AM
3c905C network card in docking station. Cannot activate network device eth0. pathelms Linux - Networking 4 04-12-2006 02:55 PM
Neural Networks Mohsen Programming 6 02-11-2004 09:25 AM
Project=IDS + neural networks A-dummy Linux - Security 1 11-02-2002 12:48 PM


All times are GMT -5. The time now is 10:27 AM.

Main Menu
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
identi.ca: @linuxquestions
Facebook: linuxquestions Google+: linuxquestions
Open Source Consulting | Domain Registration