Probabilistic-Programming-and-Bayesian-Methods-for-Hackers - aka "Bayesian Methods for Hackers": An introductio

项目说明

Bayesian Methods for Hackers

#### *Using Python and PyMC*

The Bayesian method is the natural approach to inference, yet it is hidden from readers behind chapters of slow, mathematical analysis. The typical text on Bayesian inference in two to three chapters on probability theory, then enters what Bayesian inference is. Unfortunately, due to mathematical intractability of most Bayesian models, the reader is only shown simple, artificial examples. This can leave the user with a *so-what* feeling about Bayesian inference. In fact, this was the author's own prior opinion.

After some recent success of Bayesian methods in machine-learning competitions, I decided to in the subject again. Even with my mathematical background, it took me three straight-days of reading examples and trying to put the pieces together to understand the methods. There was simply not enough literature bridging theory to practice. The problem with my misunderstanding was the disconnect between Bayesian mathematics and probabilistic programming. That being said, I suffered then so the reader would not have to now. This book attempts to bridge the gap.

If Bayesian inference is the destination, then mathematical analysis is a particular path towards it. On the other hand, computing power is cheap enough that we can afford to take an alternate route programming. The latter path is much more useful, as it denies the necessity of mathematical inter at each step, that is, we remo mathematical analysis as a prerequisite to Bayesian inference. Simply put, this latter computational path proceeds intermediate jumps from beginning to end, where as the first path proceeds by enormous leaps, often landing far away from our target. Furthermore, without a strong mathematical background, the analysis required by the first path cannot even take place.

*Bayesian Methods for Hackers* is designed as an introduction to Bayesian inference from a computational/understanding-first, and mathematics-second, point of view. Of course as an introductory book, we can only leave it at that: an introductory book. For the mathematically trained, they may cure the curiosity this text generates with other texts designed with mathematical analysis in mind. For the enthusiast with less mathematical background, or one who is not interested in the mathematics but simply the practice of Bayesian methods, this text should be sufficient and entertaining.

The choice of PyMC as the probabilistic programming language is two-fold. As of this writing, there is currently no central resource for examples and explanations in the PyMC universe. The official documentation assumes prior knowledge of Bayesian inference and probabilistic programming. We hope this book encourages users at every level to look at PyMC. Secondly, with recent core de and popularity of the scientific stack in Python, PyMC is likely to become a core component soon enough.

PyMC does ha to run, namely NumPy and (optionally) SciPy. To not limit the user, the examples in this book will rely only on PyMC, NumPy, SciPy and Matplotlib.

Printed by Addison-Wesley
------
<di="float: right; margin-left: 30px;"><img title="Bayesian Methods for Hackersg"style="float: right;margin-left: 30px;" src=" align=right height = 200 /></div>

Bayesian Methods for Hackers is now a as a printed book! You can pick up a copy on Amazon. What are the differences between the online and the printed ?

- Additional Chapter on Bayesian A/B testing
- Updated examples
- Answers to the end of chapter questions
- Additional explanation, and rewritten sections to aid the reader.

Contents
------

See the project homepage here for examples, too.

The below chapters are rendered via the *nb* at
nb.jupyter.org/, and is read-only and rendered in real-time.
Interacti + examples can be downloaded by cloning!

PyMC2

* Prologue: Why we do it.

* Chapter 1: Introduction to Bayesian Methods
Introduction to the philosophy and practice of Bayesian methods and answering the question, "What is probabilistic programming?" Examples include:
- Inferring human behaviour changes from text message rates

* Chapter 2: A little more on PyMC
We explore modeling Bayesian problems using Python's PyMC library through examples. How do we create Bayesian models? Examples include:
- Detecting the frequency of cheating students, while a liars
- Calculating probabilities of the Challenger space-shuttle disaster

* Chapter 3: Opening the Black Box of MCMC
We discuss how MCMC operates and diagnostic tools. Examples include:
- Bayesian clustering with mixture models

* Chapter 4: The Greatest Theorem Never Told
We explore an incredibly useful, and dangerous, theorem: The Law of Large Numbers. Examples include:
- Exploring a Kaggle dataset and the pitfalls of nai
- How to sort Reddit comments from best to worst (not as easy as you think)

* Chapter 5: Would you rather lose an arm or a leg?
The introduction of loss functions and their (awesome) use in Bayesian methods. Examples include:
- Solving the *Price is Right*'s Showdown
- Optimizing financial predictions
- Winning solution to the Kaggle Dark World's competition

* Chapter 6: Getting our *prior*-ities straight
Probably the most important chapter. We draw on expert opinions to answer questions. Examples include:
- Multi-Armed Bandits and the Bayesian Bandit solution.
- What is the relationship between data sample size and prior?
- Estimating financial unknowns using expert priors

We explore useful tips to be objective in analysis as well as common pitfalls of priors.

PyMC3

* Prologue: Why we do it.

* Chapter 1: Introduction to Bayesian Methods
Introduction to the philosophy and practice of Bayesian methods and answering the question, "What is probabilistic programming?" Examples include:
- Inferring human behaviour changes from text message rates

* Chapter 2: A little more on PyMC
We explore modeling Bayesian problems using Python's PyMC library through examples. How do we create Bayesian models? Examples include:
- Detecting the frequency of cheating students, while a liars
- Calculating probabilities of the Challenger space-shuttle disaster

* Chapter 3: Opening the Black Box of MCMC
We discuss how MCMC operates and diagnostic tools. Examples include:
- Bayesian clustering with mixture models

* Chapter 4: The Greatest Theorem Never Told
We explore an incredibly useful, and dangerous, theorem: The Law of Large Numbers. Examples include:
- Exploring a Kaggle dataset and the pitfalls of nai
- How to sort Reddit comments from best to worst (not as easy as you think)

* Chapter 5: Would you rather lose an arm or a leg?
The introduction of loss functions and their (awesome) use in Bayesian methods. Examples include:
- Solving the *Price is Right*'s Showdown
- Optimizing financial predictions
- Winning solution to the Kaggle Dark World's competition

* Chapter 6: Getting our *prior*-ities straight
Probably the most important chapter. We draw on expert opinions to answer questions. Examples include:
- Multi-Armed Bandits and the Bayesian Bandit solution.
- What is the relationship between data sample size and prior?
- Estimating financial unknowns using expert priors

We explore useful tips to be objective in analysis as well as common pitfalls of priors.

More questions about PyMC?
Please post your modeling, con, or any other PyMC question on cross-, the statistics stack-exchange.

Using the book
-------

The book can be read in three different ways, starting from most recommended to least recommended:

1. The most recommended option is to clone the repository to download the .ipynb files to your local machine. If you ha installed, you can view the
chapters in your browser *plus* edit and run the 代码 provided (and try some practice questions). This is the preferred option to read
this book, though it comes with some dependencies.
- Jupyter is a requirement to view the ipynb files. It can be downloaded here. Jupyter notebooks can be run by (your-) ~/path/to/the/book/Chapter1_Introduction $ jupyter notebook
- For Linux users, you should not have a problem installing NumPy, SciPy, Matplotlib and PyMC. For Windows users, check out pre-compiled if you ha.
- In the styles/ directory are a number of files (.matplotlirc) that used to make things pretty. These are not only designed for the book, but they offer many impro over the default settings of matplotlib.
2. The second, preferred, option is to use the nb.jupyter.org site, which display Jupyter notebooks in the browser (example).
The contents are updated synchronously as commits are made to the book. You can use the Contents section above to link to the chapters.

3. PDFs are the least-preferred method to read the book, as PDFs are static and non-interactive. If PDFs are desired, they can be created dynamically using the nbconvert utility.

Installation and configuration
------

If you would like to run the Jupyter notebooks locally, (option 1. above), you'll need to install the following:

Jupyter is a requirement to view the ipynb files. It can be downloaded here

Necessary packages are PyMC, NumPy, SciPy and Matplotlib.

- For Linux/OSX users, you should not have a problem installing the above, *except for Matplotlib on OSX*.
- For Windows users, check out pre-compiled if you ha.
- also recommended, for data-mining exercises, are PRAW and requests.

New to Python or Jupyter, and help with the namespaces? Check out this answer.

In the styles/ directory are a number of files that are customized for the notebook.

These are not only designed for the book, but they offer many impro over the
default settings of matplotlib and the Jupyter notebook. The in notebook style has not been finalized yet.

De
------

This book has an unusual de design. The content is open-sourced, meaning anyone can be an author.
Authors submit content or re using the GitHub interface.

How to contribute

#### What to contribute?

The current chapter list is not finalized. If you see something that is missing (MCMC, MAP, Bayesian networks, good prior choices, Potential classes etc.),

feel free to start there.

Cleaning up Python 代码 and making 代码 more PyMC-esque

Giving better explanations

Spelling/grammar mistakes

Suggestions

Contributing to the Jupyter notebook styles

#### Commiting

All commits are welcome, even if they are minor ;)

If you are unfamiliar with Github, you can email me contributions to the email below.

Reviews
------
*these are satirical, but real*

"No, but it looks good" - John D. Cook

"I ... read this book ... I like it!" - Andrew Gelman

"This book is a godsend, and a direct refutation to that 'hmph! you don't know maths, piss off!' school of thought...
The publishing model is so unusual. Not only is it open 源码 but it relies on pull requests from anyone in order to progress the book. This is ingenious and heartening" - excited Reddit user

Contributions and Thanks
-----

Thanks to all our contributing authors, including (in chronological order):

Authors | | | |
--- | --- | --- | ---
Cameron Da | Stef Gibson | Ohprecio |Lars Buitinck
Paul Magwene | Matthias Bussonnier | Jens Rantil | y-p
Ethan Brown | Jonathan Whitmore | Mattia Rigotti | Colby Lemon
Gusta | Matthew Conlen | Jim Radford | Sabino
Thomas Bratt | Nisan Haramati | Robert Grant | Matthew Wampler-Doty
Yarosla | Alex Garel | Oleksandr Lysenko | liori
ducky427 | Pablo de Oliveira Castro | sergeyfogelson | Mattia Rigotti
Matt Bauman | Andrew Duberstein | Carsten Brandt | Bob Jansen
ugurthemaster | William Scott | Min RK | Bulwersator
elpres | Augusto Hack | Michael Feldmann | Youki
Jens Rantil | Kyle Meyer | Eric Martin | Inconditus
Kleptine | Stuart Layton | Antonino Ingargiola | vsl9
Tom Christie | bclow | Simon Potter | Garth Snyder
Daniel Beauchamp | Philipp Singer | gbenmartin | Peadar Coyle

We would like to thank the Python community for building an amazing architecture. We would like to thank the
statistics community for building an amazing architecture.

Similarly, the book is only possible because of the PyMC library. A big thanks to the core devs of PyMC: Chris Fonnesbeck, Anand Patil, Da and John Sal.

One final thanks. This book was generated by Jupyter Notebook, a wonderful 工具 for de in Python. We thank the IPython/Jupyter
community for de the Notebook interface. All Jupyter notebook files are a for download on the GitHub repository.

#### Contact
Contact the main author, Cam Da at or

!Imgur

项目信息

开发语言：Jupyter Notebook

Star 数：28178

技术要点

该项目属于黑客技术方向的开源资源，代码结构清晰，适合安全研究人员学习参考。项目围绕aka "Bayesian Methods for Hackers": An introduction to Bayesian methods + probabilistic programming with a computation/understanding-first, mathematics-second point of view. All in pure Python ;) 展开，提供了可运行的实现，对理解相关安全机制、工具原理与攻防技术具有较高的研究价值。

从技术实现角度看，此类项目的核心在于对底层原理的深入理解与工程化落地。开发者在研究过程中可以结合官方文档与社区讨论，梳理项目的初始化流程、核心模块职责与数据流转，逐步掌握其中的技术细节与设计思路。

使用与学习建议

1. 下载源码压缩包，解压到本地环境。

2. 根据项目类型准备对应的运行环境与依赖。

3. 按照 README 中的步骤完成配置与部署。

4. 在隔离的测试环境中运行验证，避免影响生产系统。

免责声明

本资源来源于开源社区，仅供安全研究、技术学习与合法授权范围内的攻防演练使用，请勿用于任何非法或侵害他人权益的用途。使用本项目产生的一切后果由使用者自行承担。