of 6
All materials on our website are shared by users. If you have any questions about copyright issues, please report us to resolve them. We are always happy to assist you.
Desenvolvido pelo MIT
  1 Bayesian regression and Bitcoin Devavrat Shah Kang ZhangLaboratory for Information and Decision SystemsDepartment of EECSMassachusetts Institute of Technology, Abstract  —In this paper, we discuss the methodof Bayesian regression and its efficacy for predictingprice variation of Bitcoin, a recently popularizedvirtual, cryptographic currency. Bayesian regres-sion refers to utilizing empirical data as proxy toperform Bayesian inference. We utilize Bayesianregression for the so-called “latent source model”.The Bayesian regression for “latent source model”was introduced and discussed by Chen, Nikolov andShah [1] and Bresler, Chen and Shah [2] for the purpose of binary classification. They establishedtheoretical as well as empirical efficacy of themethod for the setting of binary classification.In this paper, instead we utilize it for predictingreal-valued quantity, the price of Bitcoin. Based onthis price prediction method, we devise a simplestrategy for trading Bitcoin. The strategy is ableto nearly double the investment in less than 60 dayperiod when run against real data trace. I. Bayesian Regression The problem.  We consider the question of regres-sion: we are given  n  training labeled data points( x i ,y i ) for 1  ≤  i  ≤  n  with  x i  ∈  R d ,y i  ∈  R  for somefixed  d  ≥  1. The goal is to use this training data topredict the unknown label  y  ∈ R  for given  x ∈ R d . The classical approach.  A standard approach fromnon-parametric statistics (cf. see [3] for example) is toassume model of the following type: the labeled datais generated in accordance with relation  y  =  f  ( x ) +   where    is an independent random variable repre-senting noise, usually assumed to be Gaussian withmean 0 and (normalized) variance 1. The regressionmethod boils down to estimating  f   from  n  observation( x 1 ,y 1 ) ,..., ( x n ,y n ) and using it for future prediction.For example, if   f  ( x ) =  x T  θ ∗ , i.e.  f   is assumed tobe linear function, then the classical least-squaresestimate is used for estimating  θ ∗ or  f  :ˆ θ LS   ∈ argmin θ ∈ R d n  i =1 ( y i − x T i  θ ) 2 (1)In the classical setting,  d  is assumed fixed and  n  d which leads to justification of such an estimator beinghighly effective. In various modern applications,  n  d or even  n    d  is more realistic and thus leavinghighly under-determined problem for estimating  θ ∗ .Under reasonable assumption such as ‘sparsity’ of   θ ∗ ,i.e.   θ ∗  0    d , where   θ ∗  0  =  |{ i  :  θ ∗ i   = 0 }| ,the regularized least-square estimation (also knownas Lasso [4]) turns out to be the right solution: forappropriate choice of   λ >  0,ˆ θ LASSO  ∈ argmin θ ∈ R d n  i =1 ( y i − x T i  θ ) 2 +  λ  θ  1 .  (2)At this stage, it is worth pointing out that theabove framework, with different functional forms, hasbeen extremely successful in practice. And very excit-ing mathematical development has accompanied thistheoretical progress. The book [3] provides a goodoverview of this literature. Currently, it is a very activearea of research. Our approach.  The key to success for the abovestated approach lies in the ability to choose a reason-able parametric function space over which one triesto estimate parameters using observations. In variousmodern applications (including the one considered inthis paper), making such a choice seems challenging.The primary reason behind this is the fact that thedata is very high dimensional (e.g. time-series) makingeither parametric space too complicated or meaning-less. Now in many such scenarios, it seems that thereare few prominent ways in which underlying eventexhibits itself. For example, a phrase or collection of words become viral on Twitter social media for fewdifferent reasons – a public event, life changing eventfor a celebrity, natural catastrophe, etc. Similarly,there are only few different types of people in termsof their choices of movies – those of who like comediesand indie movies, those who like court-room dramas,etc. Such were the insights formalized in works [1]and [2] as the ‘latent source model’ which we describeformally in the context of above described framework.   a  r   X   i  v  :   1   4   1   0 .   1   2   3   1  v   1   [  c  s .   A   I   ]   6   O  c   t   2   0   1   4  2 There are  K   distinct latent sources  s 1 ,...,s K   ∈ R d ;a latent distribution over  { 1 ,...,K  }  with associatedprobabilities { µ 1 ,...,µ K  } ; and  K   latent distributionsover  R , denoted as  P 1 ,..., P K  . Each labeled datapoint ( x,y ) is generated as follows. Sample index  T   ∈{ 1 ,...,K  }  with  P ( T   =  k ) =  µ k  for 1  ≤  k  ≤  K  ;  x  = s T   +   , where    is  d -dimensional independent randomvariable, representing noise, which we shall assume tobe Gaussian with mean vector  0  = (0 ,..., 0) ∈ R d andidentity covariance matrix;  y  is sampled from R as perdistribution  P T  .Given this model, to predict label  y  given associatedobservation  x , we can utilize the conditional distribu-tion 1 of   y  given  x  given as follows: P  y  x   = T   k =1 P  y  x,T   =  k  P  T   =  k  x  ∝ T   k =1 P  y  x,T   =  k  P  x  T   =  k  P ( T   =  k )= T   k =1 P k  y  P    = ( x − s k )  µ k = T   k =1 P k  y  exp  −  12  x − s k  22  µ k .  (3)Thus, under the latent source model, the prob-lem of regression becomes a very simple Bayesianinference problem. However, the problem is lack of knowledge of the ‘latent’ parameters of the sourcemodel. Specifically, lack of knowledge of   K  , sources( s 1 ,...,s K  ), probabilities ( µ 1 ,...,µ K  ) and probabil-ity distributions  P 1 ,..., P K  .To overcome this challenge, we propose the follow-ing simple algorithm: utilize empirical data as proxyfor estimating conditional distribution of   y  given  x given in (3). Specifically, given  n  data points ( x i ,y i ),1 ≤ i ≤ n , the empirical conditional probability is P emp  y  x   =  ni =1 1 ( y  =  y i )exp  −  14  x − x i  22  ni =1  exp  −  14  x − x i  22   . (4)The suggested empirical estimation in (4) has thefollowing implications: in the context of binary clas-sification,  y  takes values in  { 0 , 1 } . Then (4) suggests 1 Here we are assuming that the random variables havewell-defined densities over appropriate space. And when ap-propriate, conditional probabilities are effectively representingconditional probability density. the following classification rule: compute ratio P emp  y  = 1  x  P emp  y  = 0  x  =  ni =1 1 ( y i  = 1)exp  −  14  x − x i  22  ni =1 1 ( y i  = 0)exp  −  14  x − x i  22  .  (5)If the ratio is  >  1, declare  y  = 1, else declare  y  = 0.In general, to estimate the conditional expectation of  y , given observation  x , (4) suggests E emp [ y | x ] =  ni =1  y i  exp  −  14  x − x i  22  ni =1  exp  −  14  x − x i  22   .  (6)Estimation in (6) can be viewed equivalently as a‘linear’ estimator: let vector  X  ( x ) ∈ R n be such that X  ( x ) i  = exp   −  14  x  −  x i  22  /Z  ( x ) with  Z  ( x ) =  ni =1  exp   −  14  x  −  x i  22  , and  y  ∈  R n with  i thcomponent being  y i , then ˆ y  ≡ E emp [ y | x ] isˆ y  =  X  ( x ) y .  (7)In this paper, we shall utilize (7) for predicting futurevariation in the price of Bitcoin. This will further feedinto a trading strategy. The details are discussed inthe Section II. Related prior work.  To begin with, Bayesian in-ference is foundational and use of empirical data asa proxy has been a well known approach that ispotentially discovered and re-discovered in variety of contexts over decades, if not for centuries. For exam-ple, [5] provides a nice overview of such a method for aspecific setting (including classification). The concreteform (4) that results due to the assumption of latentsource model is closely related to the popular rulecalled the ‘weighted majority voting’ in the literature.It’s asymptotic effectiveness is discussed in literatureas well, for example [6].The utilization of latent source model for thepurpose of identifying precise sample complexity forBayesian regression was first studied in [1]. In [1], authors showed the efficacy of such an approach forpredicting trends in social media Twitter. For thepurpose of the specific application, authors had toutilize noise model that was different than Gaussianleading to minor change in the (4) – instead of usingquadratic function, it was quadratic function appliedto logarithm (component-wise) of the underlying vec-tors - see [1] for further details.In various modern application such as online recom-mendations, the observations ( x i  in above formalism)  3 are only partially observed. This requires further mod-ification of (4) to make it effective. Such a modifica-tion was suggested in [2] and corresponding theoreticalguarantees for sample complexity were provided.We note that in both of the works [1], [2], the Bayesian regression for latent source model was usedprimarily for binary classification. Instead, in thiswork we shall utilize it for estimating real-valuedvariable. II. Trading Bitcoin What is Bitcoin.  Bitcoin is a peer-to-peer crypto-graphic digital currency that was created in 2009 byan unknown person using the alias Satoshi Nakamoto[7], [8]. Bitcoin is unregulated and hence comes with benefits (and potentially a lot of issues) such astransactions can be done in a frictionless manner - nofees - and anonymously. It can be purchased throughexchanges or can be ‘mined’ by computing/solvingcomplex mathematical/cryptographic puzzles. Cur-rently, 25 Bitcoins are rewarded every 10 minutes(each valued at around US $400 on September 27,2014). As of September 2014, its daily transactionvolume is in the range of US $30-$50 million andits market capitalization has exceeded US $7 billion.With such huge trading volume, it makes sense tothink of it as a proper financial instrument as part of any reasonable quantitative (or for that matter any)trading strategy.In this paper, our interest is in understandingwhether there is ‘information’ in the historical datarelated to Bitcoin that can help predict future pricevariation in the Bitcoin and thus help develop prof-itable quantitative strategy using Bitcoin. As men-tioned earlier, we shall utilize Bayesian regressioninspired by latent source model for this purpose. Relevance of Latent Source Model.  Quantitativetrading strategies have been extensively studied andapplied in the financial industry, although many of them are kept secretive. One common approach re-ported in the literature is technical analysis, whichassumes that price movements follow a set of patternsand one can use past price movements to predictfuture returns to some extent [9], [10]. Caginalp and Balenovich [11] showed that some patterns emergefrom a model involving two distinct groups of traderswith different assessments of valuation. Studies foundthat some empirically developed geometric patterns,such as heads-and-shoulders, triangle and double-top-and-bottom, can be used to predict future pricechanges [12], [13], [14]. The Latent Source Model is precisely trying tomodel existence of such underlying patterns leadingto price variation. Trying to develop patterns withthe help of a human expert or trying to identifypatterns explicitly in the data, can be challenging andto some extent subjective. Instead, using Bayesianregression approach as outlined above allows us toutilize the existence of patterns for the purpose of better prediction without explicitly finding them. Data.  In this paper, to perform experiments, we haveused data related to price and order book obtainedfrom  – one of the largest exchangesoperating in China. The data concerns time periodbetween February 2014 to July 2014. The total rawdata points were over 200 million. The order book dataconsists of 60 best prices at which one is willing tobuy or sell at a given point of time. The data pointswere acquired at the interval of every two seconds.For the purpose of computational ease, we constructeda new time series with time interval of length 10seconds; each of the raw data point was mapped to theclosest (future) 10 second point. While this coarseningintroduces slight ‘error’ in the accuracy, since ourtrading strategy operates at a larger time scale, thisis insignificant. Trading Strategy.  The trading strategy is verysimple: at each time, we either maintain positionof +1 Bitcoin, 0 Bitcoin or  − 1 Bitcoin. At eachtime instance, we predict the average price movementover the 10 seconds interval, say ∆  p , using Bayesianregression (precise details explained below) - if ∆  p > t ,a threshold, then we buy a bitcoin if current bitcoinposition is  ≤  0; if ∆  p <  − t , then we sell a bitcoin if current position is  ≥  0; else do nothing. The choiceof time steps when we make trading decisions asmentioned above are chosen carefully by looking atthe recent trends. We skip details as they do not havefirst order effect on the performance. Predicting Price Change.  The core method foraverage price change ∆  p  over the 10 second intervalis the Bayesian regression as in (7). Given time-series of price variation of Bitcoin over the intervalof few months, measured every 10 second interval, wehave a very large time-series (or a vector). We usethis historic time series and from it, generate threesubsets of time-series data of three different lengths: S  1  of time-length 30 minutes,  S  2  of time-length 60minutes, and  S  3  of time-length 120 minutes. Now at agiven point of time, to predict the future change ∆  p ,we use the historical data of three length: previous30 minutes, 60 minutes and 120 minutes - denoted  4 x 1 ,x 2 and  x 3 . We use  x  j with historical samples  S   j for Bayesian regression (as in (7)) to predict averageprice change ∆  p  j for 1  ≤  j  ≤  3. We also calculate r  = ( v bid − v ask ) / ( v bid + v ask ) where  v bid  is total volumepeople are willing to buy in the top 60 orders and  v ask is the total volume people are willing to sell in the top60 orders based on the current order book data. Thefinal estimation ∆  p  is produced as∆  p  =  w 0  + 3   j =1 w  j ∆  p  j +  w 4 r,  (8)where  w  = ( w 0 ,...,w 4 ) are learnt parameters. Inwhat follows, we explain how  S   j ,  1  ≤  j  ≤  3 arecollected; and how  w  is learnt. This will complete thedescription of the price change prediction algorithmas well as trading strategy.Now on finding  S   j , 1  ≤  j  ≤  3 and learning  w . Wedivide the entire time duration into three, roughlyequal sized, periods. We utilize the first time period tofind patterns  S   j ,  1 ≤  j  ≤ 3. The second period is usedto learn parameters w  and the last third period is usedto evaluate the performance of the algorithm. Thelearning of   w  is done simply by finding the best linearfit over all choices given the selection of   S   j ,  1 ≤  j  ≤ 3.Now selection of   S   j ,  1  ≤  j  ≤  3. For this, we take allpossible time series of appropriate length (effectivelyvectors of dimension 180, 360 and 720 respectivelyfor  S  1 ,S  2  and  S  3 ). Each of these form  x i  (in thenotation of formalism used to describe (7)) and theircorresponding label  y i  is computed by looking at theaverage price change in the 10 second time intervalfollowing the end of time duration of   x i . This datarepository is extremely large. To facilitate computa-tion on single machine with 128 G  RAM with 32 cores,we clustered patterns in 100 clusters using  k − meansalgorithm. From these, we chose 20 most effectiveclusters and took representative patterns from theseclusters.The one missing detail is computing ‘distance’ be-tween pattern  x  and  x i  - as stated in (7), this issquared   2 -norm. Computing   2 -norm is computation-ally intensive. For faster computation, we use negativeof ‘similarity’, defined below, between patterns as’distance’. Definition 1.  (Similarity) The similarity betweentwo vectors  a ,  b ∈ R M  is defined as s ( a , b ) =  M z =1 ( a z − mean ( a ))( b z − mean ( b )) M std ( a )  std ( b )  ,  (9)where  mean ( a ) = (  M z =1  a z ) /M   (respectively for  b )and  std ( a ) = (  M z =1 ( a i − mean ( a )) 2 ) /M   (respectivelyfor  b ).In (7), we use exp( c · s ( x,x i )) in place of exp( − x − x i  22 / 4) with choice of constant  c  optimized for betterprediction using the fitting data (like for  w ).We make a note of the fact that this similaritycan be computed very efficiently by storing the pre-computed patterns (in  S  1 ,S  2  and  S  3 ) in a normalizedform (0 mean and std 1). In that case, effectivelythe computation boils down to performing an inner-product of vectors, which can be done very efficiently.For example, using a straightforward Python imple-mentation, more than 10 million cross-correlations canbe computed in 1 second using 32 core machine with128 G  RAM.Fig. 1: The effect of different threshold on the numberof trades, average holding time and profitFig. 2: The inverse relationship between the averageprofit per trade and the number of trades Results.  We simulate the trading strategy describedabove on a third of total data in the duration of May6, 2014 to June 24, 2014 in a causal manner to see howwell our strategy does. The training data utilized is allhistorical (i.e. collected before May 6, 2014). We usedifferent threshold  t  and see how the performance of strategy changes. As shown in Figure 1 and 2 different threshold provide different performance. Concretely,as we increase the threshold, the number of trades
We Need Your Support
Thank you for visiting our website and your interest in our free products and services. We are nonprofit website to share and download documents. To the running of this website, we need your help to support us.

Thanks to everyone for your continued support.

No, Thanks