In case you haven’t heard by now heads up limit hold’em has been solved. You can read the original scientific paper here
You can query the GTO bot named Cepheus’ strategy here http://poker.srv.ualberta.ca/strategy
But it’s slow and not really displayed in an easy to understand way for humans emulate. It’s not easy to figure out if it tends to check back certain boards or what its cbet % is for example. I was curious if you could download the code and run it yourself and how much it would cost. The good news is you can download and run it yourself, the bad news it’s going to cost you a lot to get an exact Cepheus replica($500k by my estimate).
Cost of computing estimate
Looking at http://aws.amazon.com/ec2/pricing/
A large compute optimized on-demand instance with 32 2.6-GHz Intel Xeon cores, 60 GB of RAM, and 320-GB of local disk is probably the most similar node to the ones used by the Alberta researchers. They cost $1.68 per hour. To run 200 of them for 68.5 days would cost you 200*1.68*24*68.5=$552,384
Of course computing cost decreases over time over time as shown by the graph below. In a five years this computation cost should be one tenth of this current estimate and we’ll have 10TB hardrives for $50(current cost of 1TB hardrives http://www.amazon.com/Seagate-Desktop-3-5-Inch-Internal-ST1000DM003/dp/B005T3GRNW) and be able to download and save the whole strategy easily on our home computer.
Scientific Paper Interesting Stats
There were a few interesting stats given in the paper that I don’t feel were talked about much by the media because it wasn’t explained all that well and put in normal “poker speak” terms. They referred to a “hand” as a “game” and measured edges in “milli-big-blinds/game” instead of a more typical “bb/100″.
The maximum achievable winrate playing vs Cepheus is listed in the paper as 0.986 milli-big-blind per game(by game they mean hand). In poker terms this equates to (0.986 milli big blinds / hand) * (1 big blind / 1000 milli big blinds) * (100) = 0.986 * .1 = 0.0986 bb/100 maximum winrate. So for example if you were playing true GTO headsup $200/$400 limit holdem vs Cepheus you could expect to win about $200 * 0.0986 = $19.72 per hour per table.
Some major troll nits were complaining that if it’s possible to have a positive winrate vs Cepheus then heads up limit hold’em isn’t solved. What I would say to these people is
a) 0.1 bb/100 achievable winrate is tiny and
b) if they really want to get this number lower they can do so by simply turning their algorithm back on and letting it run more iterations.
It lists the button’s winrate vs itself as “between 87.7 and 89.7 mbb/g for the dealer” This means the GTO edge for the button vs the big blind is 88 * 0.1 = 8.8 bb/100. So in other words when someone hit and runs one hand with their button vs you they’re stealing 0.088bb or $17.6 at $200/$400 in EV from you.
Reducing computation cost
There are a number of ways you could create a much cheaper “good enough” GTO bot yourself. The way that they created Cepheus is by running this code http://webdocs.cs.ualberta.ca/~games/poker/software/CFR_plus.tar.bz2 on a 200 node cluster of computers for 68 days. It is described in the paper as
Our CFR+ implementation was executed on a cluster of 200 computation nodes each with 24 2.1-GHz AMD cores, 32GB of RAM, and a 1-TB local disk. We divided the game into 110,565 subgames (partitioned according to preflop betting, flop cards, and flop betting). The subgames were split among 199 worker nodes, with one parent node responsible for the initial portion of the game tree. The worker nodes performed their updates in parallel, passing values back to the parent node for it to perform its update, taking 61 minutes on average to complete one iteration. The computation was then run for 1579 iterations, taking 68.5 days, and using a total of 900 core-years of computation (43) and 10.9 TB of disk space, including filesystem overhead from the large number of files.
If you simply reduced the number of iterations ran you could create a not quite as good bot for a fraction of the cost. See the figure below from the scientific paper. Since they ran their sim for 900 core years or 1579 iterations they achieved a maximum exploitability of ~0.1 bb/100 (~$500k computation cost). Interpolating this graph that means that in 90 core-years of computation you could create a a bot with 1 bb/100 of exploitablity (~$50k computation cost). After 27 core-years computation cost you could create a bot with 10 bb/100 of exploitability (~$17k computation cost). After 9 core-years computation cost you could create a bot with 30 bb/100 of exploitability (~$5k computation cost).
You could further reduce computation cost by reducing the number of subgames. They solved for 110,565 subgames and their preflop strategy is very easy to view and download here http://poker.srv.ualberta.ca/preflop. You could hard code this in and reduce your computation cost drastically. Unfortunately at this time I haven’t worked out the math on how they arrive at 110,565 to calculate exactly what order of magnitude of computation this may save. If someone could help me out with that it would be greatly appreciated. There’s only 169 different preflop hand combinations, 1755 different flops each with 47 different turn cards and 46 different river cards.
Other games and online poker’s future
Their algorithm could easily be adapted to other limit games. Headsup limit Omaha 8 or better should be considered solved as well at this point. If someone wants to give me $1 million I’ll prove it. Same goes for headsup 2-7 triple draw. The stud games have a much higher number of game states so they may not be cheaply solvable at this point. Razz on the other hand ignores suits and only has 13 unique cards so a “good enough” headsup GTO bot (one with say 1bb/100 exploitability) could probably be created for $50k or less in computation cost at this point. It’s all a matter of time before all the games are dead and solved.
Some people will point out that the game state size of a no limit game is huge and may never be solved in our lifetime.
Sure, it’s true that we may not see a Headsup No Limit Hold’em bot with maximum exploitability of less than 0.1bb/100 in our lifetime. That does not mean a “good enough” no limit bot with artificial pot size bet constraints (can only bet pot, 1/2 pot, 1/4 pot for instance) with less than 1 bb/100 of maximum exploitability could not be created TODAY for a couple hundred thousand.
It’s just a race to see who can figure it out first. The nosebleed guys hire programmers to figure out things out for them. They have the most money, resources and incentive to do so. It’s no secret at this point. I’m not optimistic about online poker’s future. Nobody plays online chess for money. Then again they do play online blackjack.
A more plain english summary of the paper is available here
You can play against Cepheus here
1. Manage my time better. This is my main goal and all other goals are really a subset of this goal.
2. Wake up and go to bed at regular times. I’ve been self employed for over six years now and waking up early and regularly is something I’ve struggled with as long as I can remember. Waking up late prevents me from having a very productive day and achieving my goals quicker than I otherwise would if I woke up early. Over the years I’ve learned a couple tricks to help me like practicing waking up, going to bed early the night before, and most of all doing it regularly since it takes about a week for your sleep schedule to adjust. Hopefully I can change that in 2014.
3. Read less 2+2/reddit. I’ve grown into a schedule of browsing the internet for an hour or two when I first wake up and right before I go to bed. Sometimes I read and learn interesting things, but usually I just end up reading random semi entertaining threads like If a player has a prosthetic arm that he used for holding onto the football and the arm got detached during the play, would that be considered a fumble or would the prosthetic limb still have possession for the player? In the morning it stops me from being productive, and in the evening it tends to keep me up later than I want to stay up and prevents me from being productive the next day. I don’t think I’ll ever kick the habit for good, but I’d like to decrease the amount of time I spend randomly browsing to focus on more productive or entertaining things.
4. Read more books. I’m somewhat embarrassed to say I only read 1.5 books last year. This is the least number I’ve read in a year since I was probably 12. I read American Sniper: The Autobiography of the Most Lethal Sniper in U.S. Military History and half of What Every BODY is Saying: An Ex-FBI Agent’s Guide to Speed-Reading People Reading novels has been shown to boost brain function for days It increases your vocabulary and provides much deeper explanation on topics that you just can’t get from tweets and short internet articles which takes up the majority of young people’s reading sources these days. Plus at 1.5 books/year I’m getting dangerously close to becoming like Kanye West:
5. Exercise more. Pretty standard resolution here. Three times a week is what I’m shooting for. Maybe I will finally run that marathon this year I’ve been wanting to for awhile now, but we’ll see.
6. Blog at least once a month. I started tweeting a bit in 2013 and it kind of replaced my urge to blog, but there’s just some things you can’t get out in 140 characters. I don’t think 12 blogs per year is asking too much of my time.
7. Watch less TV. Spend more time reading books and working out instead.
8. Grow my businesses. Buy a house. These are pretty vague goals which are generally bad for goal making, but I’ll take these more on a month to month basis. In January I’d like to focus on BitcoinRichList.com and then on poker for Australia and the Aussie millions. In February I’d like to focus more on ATMs and condo buying.
Happy new year and best of luck to you all in 2014!
Follow me @andr3w321 I’ve been at it for a couple weeks now and it’s much less time consuming and easier to do at the table while I’m playing and doesn’t cut into sleeping or eating time.
I made day 2 of the main event with 28k in chips, play resumes tomorrow. I bricked every wsop tourny I played, but I did get four cashes in Carnivale events and am currently tied as the TLB leader here: http://www.wsop.com/players/2013.asp My housemate Matt Ashton has seriously one upped me however and is the current wsop player of the year leader! I’ll probably write a proper wsop wrapup at the end of the series, but for now I’m focused on playing and resting. Best of luck to anyone else still in the main!
I made day 2 of the $2,500 8 game. My chips aren’t great, about half average at 8k, but I’m still in it. 40 get paid and 192 people are left. I had quite a bit of nerves for the first two hours of play or so since I haven’t played live in year, but after that they went away and after four hours of play I started really feeling comfortable and confident again.
My table draw was pretty tough. I had Daniel Negreanu, David Williams and four other fairly tough guys at my table. There was one live one, some russian guy obv, but he busted halfway through the day. People seemed to play alright, but some are horrible at O8 and Triple Draw. In triple draw, I saw people coldcall and draw 3, open 348 utg, stand pat with a Ten to start etc.
The most interesting hand I played was vs Negreanu. I came very close to busting Negreanu, but it was not meant to be. In PLO, the cutoff opened and I cold called AsKc8s3c on the button, Negreanu defended his big blind. Flop came down 974 with two spades and Negreanu leads out for 2/3 pot. Cutoff folds, and I ask Negreanu how much he has left. He was pretty toned down from what you see on TV for most of the day, but in this hand he really came alive.
“About 2100. You want to put me in? You can put me in if you want? Go ahead.”
I normally don’t get intimidated by pros, but this was still like level 4 or so and I haven’t played poker in awhile. He has a very big table presence when he wants to and he makes it hard to think, I almost did what he wanted and put it in right there. Instead, I end up calling. He starts bantering again,
“Now you’re going make me think. Why can’t you just put it in and make my life easy?”
The turn was an offsuit 8. He shoves and I think for a bit. He starts talking again,
“What do you have JT or something like that? You should probably call. Look at this guy. Probably doing math or something and just wants me to shutup.”
I reply “pretty much” Because I am doing math and trying to figure out pot odds and my equity. With my pair plus flush draw have equity to call if he just has 97, but I’m drawing pretty badly if he has a straight. I tank for a bit and end up calling, but his flopped 3 pair holds up and he doubles through me.
I got here last night. The flight from Seattle to Vegas is super easy. Direct and only two hours. I’m happy to say my roommate Matt Ashton is already deep in the $2,500 Stud 8/Omaha 8 or better event. Wish him luck! There’s 19 left and he’s currently 9th in chips. I’m playing the 8 game event later today and just trying to relax and chill by the pool til then. Hopefully my poker skill aren’t TOO rusty or they come back to me quickly.
Title says it all. I’m heading out to Vegas next week for my annual wsop pilgrimage. I’ll try and blog about every event I play like I do every year so check back for updates! I’m planning on playing ten to fifteen events, more than I’ve ever played. Last year I only played seven including the main event.
I’ll be there for thirty days in June. Four of these days my fiancee will be visiting so I won’t play any during that time. I’m not rich/crazy enough to play two events in one day so if I busted out on day one of every event I played the max I could play is ~25 events. Add to this the fact that I won’t play anything over a $3k buyin, I’m sure I’ll spend some days drinking by the pool and I don’t really like the $1k events or PLO events(both have too shallow starting stacks IMO). I’ll probably end up playing ten to fifteen events. It seems like every other year you hear about a guy who never played PLO in his life ended up winning the $1.5k PLO. This is a pretty good indicator this event is more luck than skill in my opinion.
I figured I’d do some math so I could come in with realistic expectations for my results. Last year I cashed two out of my seven events and clearly ran better than expected. I figure a good cash % is around 15% and a good final table % is around 2.5%, but these are clearly just guesses. Based on these numbers, if I play ten events I can expect
To cash in 1-2 events (10*.15=1.5)
I have a 25% chance of final tabling a single event (10*.025=.25)
There is a 20% chance I don’t cash in any events (.85^10=.2)
On average I should expect to go about four events before my first cash (.85^4=.52)
If I play fifteen events, I can expect
To cash in 2 events (15*.15=2.25)
I have a 37.5% chance of final tabling a single event (15*.025=.375)
There is a 9% chance I don’t cash in any events (.85^15=.09)
On average I should still expect to go about four events before my first cash
I’d REALLY like to make my first final table this year, but clearly the odds are not in my favor. More volume definitely translates into a better expectations, but honestly if I go 0-10 I may call it quits. Losing $20k straight is never fun. I’ve been trying to work out regularly and doing my best to play some bitcoin poker and play money poker on pokerstars in preparation. I have no doubt I’m a shell of the player I once was a couple years ago, but I would like to think I have more experience and am wiser for it even if some of my raising/calling ranges aren’t as profitable as they once were. I still like my odds vs all the $1.5k wsop donks. Best of luck at the wsop. I’m certainly looking forward to it and hope you are too!
I’ve been following bitcoins pretty closely for the past ten days now and the market has been insanely volatile and highly entertaining. Here’s the chart
It’s gone from $50 to $260 all the way back to $60 in the past thirty days. I first heard about bitcoins a couple years ago back in 2011. I got a couple free ones from the bitcoin faucet and played around with them for a bit. I thought they were interesting, but ultimately useless unless they gained widespread adoption. Well lately, as a result of the crazy price increases they’ve been all over the news and some legitimate merchants have started accepting them. Some people think it’s a bubble, some call it a pyramid scheme, some think they’ll be worth hundreds of thousands of dollars one day. Whatever your opinion it’s been a wild ride lately. The TV news interviews you see about them are obviously done by people completely clueless about them, however, and they always have these two very basic questions so for you newbies out there I’ll go ahead and answer them for you.
What are bitcoins?
Bitcoins are basically just numbers stored in a database(blockchain) linked to a bitcoin address. Similarly, dollars are just numbers stored in a database linked to a bank account.
Can’t some hacker just create a bunch of bitcoins?
No. Creating bitcoins requires a tremendous amount of computing power. Just how no human can multiply two 100 digit numbers in their head, a single computer can’t just magically create a bunch of bitcoins.
Differences between dollars and bitcoins
1. Fixed supply
Only 21 million bitcoins will ever exist. The number of US dollars that can ever exist is infinity.
2. Money supply over time
The numbers of both currencies is constantly changing, however, bitcoins follows a fixed predictable schedule. As of April 11, 2013 there are 11,022,500 bitcoins and something like 10 trillion US dollars. In 2014 there will be 11,812,500 bitcoins. In 2020 there will be 17,718,750 bitcoins. I have no idea how many US dollars will exist next year, or ten years from now. Nobody does. The federal reserve controls and manipulates the money supply daily. They increase or decrease the money supply by changing bank reserve requirements, buying or selling T-bills, or changing the discount rate.
3. Issuing authority
All dollars are ultimately created by the federal reserve. All bitcoins are created by thousands of individual bitcoin miners. No one miner is in control. Right now about 3600 new bitcoins are created a day. Miners do pool their resources into various mining pools, but no miner pool can ever reach 51% of the hash power or there will be problems.
Pros of bitcoins
1. 24/7 Speedy Service
Bitcoin transactions can happen anywhere, anytime and be confirmed within 10 minutes usually. Normal banks are open 9am-5pm Monday through Friday and transactions usually take 1-2 business days to become confirmed.
2. Low Transaction Fees
Bitcoins transaction fees are 0.0005 BTC per transaction(okay not exactly, but pretty much, click the link for more details). At $100 USD/BTC that’s equivalent to five cents. A normal credit card transaction will cost the merchant a 3% fee with a minimum fee of 25 cents. A typical wire transfer costs $25 send and sometimes a fee on the receiving end as well.
3. Everyone has a merchant account
If you want to start accepting bitcoins all you need is a bitcoin address. You don’t have to be approved by a credit card company and pay them fees once you are accepted. It’s as simple as pasting people your bank account number(bitcoin address) and you’re good to go.
4. Ease of transport
If you want to carry $20k in cash with you to Vegas it’s risky. You can lose it, someone can rob you, and authorities can confiscate it and there’s not much you can do about it. Just ask Viffer With bitcoins you just need to remember a password and that’s all you really need.
5. No Chargebacks
If you sell an item on ebay and accept paypal money, confirm the shipment, but the customer says they never received it, or lies and says it’s broken and paypal will issue a refund and you are out your money or item and there’s not much you can do about it. Bitcoins are one way transactions and irreversible(pretty much).
Cons of bitcoins
1. No Chargebacks
If someone steals your credit or debit card number you can call up your bank and get your fraudulent transactions reversed. If someone steals your bitcoin wallet there’s not much you can do. That’s why keeping your bitcoin private key secure and encrypting your wallet is so important.
2. You can delete bitcoins
Sure you can burn dollars too, but no one accidentally does that. If you lose your wallet your money is gone. Ever had a hardrive crash? Ever accidentally deleted a file you weren’t supposed to? Backups are important.
3. If your computer gets hacked you can lose all your money
If you visit the wrong website or open the wrong e-mail attachment you can lose all the money in your account. This is very scary stuff. Backups and security are your friend. Bank level security is wonderful and has had many, many years to mature. Bitcoins have existed for four years now. It’s going to take awhile(maybe never) for systems to emerge for the average joe to feel comfortable putting money into bitcoins.
4. Not many merchants accept bitcoins
This may change in the future, and new merchants sign up daily, but right now if you want to buy something with bitcoins you pretty much have to convert bitcoins to USD which makes the whole thing pretty pointless.
5. High volatility
Price stability is important for bitcoins to succeed in the long run. No one wants their money to be worth 50% of what it was worth yesterday. The US dollar fluctuates maybe 3% a year vs most other currencies. In the short run speculation will drive price, but bitcoin will need to eventually become lower volatility for it to ever work as a legitimate currency.
Due to the fixed supply of bitcoins they are susceptible to hoarding. If their coins are worth more tomorrow then they are today why would people spend them or invest them in other commodities? Well, US dollars used to be based on the gold standard and there is a fixed amount of gold on the planet. People still spent US dollars just fine. It’s all a matter of how much the value of BTC increase over time. A deflationary spiral is definitely possible and could create bubbles like we’re possibly seeing now, which could ultimately undermine confidence in bitcoins as a currency.
Warning: Don’t buy bitcoins unless you are prepared to lose all your money. They are highly volatile right now and you have to be very careful you don’t lose them or get hacked.
Largest online exchange MtGox.com
Smaller Online exchange BitStamp.net
Trade cash for bitcoins from locals LocalBitcoins.com
The easiest, quickest and cheapest method will vary depending on what site/method you use. For an online exchange the quickest and easiest is probably to do an international wire transfer directly to them. It will cost you ~$25 + receiving end fees. You can also try going through Bitinstant (Reviews are mixed) or Dwolla (takes awhile to get verified and link your account)
Seals with Clubs
-EV casino gambling
Don’t even bother
Donate to me: 18m2XtF55Yu5XZ14KakSS6ZrcdsVoSkHgb
Well it’s been a busy two weeks. Obviously I didn’t update throughout the weekend like I was hoping but here’s a rundown of what happened.
Friday: People pitched maybe 50 ideas. All the ideas were put up on the wall and then we voted on the best 20 or so and formed teams around them. We then all went home by 10pm as the building was closing.
Saturday: This was the major workday. I got there a bit before 9am and we worked for 12+ hours until the building closed again at 10pm. A few people never showed up for the next day and a couple teams went by the wayside. I guess this was inevitable, but it’s kind of oddly funny seeing the teams shrink over the weekend. We probably had one of the biggest teams. We started with around 10 and by Satuday it was down to 8. We also had a lot of developers. 6 of us wrote code and the guy whose idea it was and a microsoft PM on our team worked on the market research validation side. I certainly learned a lot as it was the first time I’d really coded in a group. The fact that we had so many devs was both a blessing and a curse in my opinion. We spent most of our time coding and actually got a working demo going where as most of the other teams just had mockups or non working html files with no backend, but I think we were weak on the presentation and market validation side of things.
Sunday: This was presentation day and most of our time was spent meeting with advisors, preparing the presentation and finishing up some coding. Unfortunately we didn’t win. That award went to corki a wine menu rating converter app. If you’re interested you can see the pitches here Our pitch was around the one hour mark.
Here’s a short highlight video of the event where I make a cameo around the 10 second mark.
The site is live right now tonightsplaylist.com It doesn’t look too different from the initial launch right now. Our team has met once since the event and made a bunch of backend code changes that you can’t see. Overall it was a great learning experience and I’d definitely do another one.