Documentation

TestingLowerBounds.Divergences.Renyi

Rényi divergence #

Main definitions #

Main statements #

Notation #

Implementation details #

theorem ProbabilityTheory.exp_mul_llr {α : Type u_1} {mα : MeasurableSpace α} {μ : MeasureTheory.Measure α} {ν : MeasureTheory.Measure α} {a : } [MeasureTheory.SigmaFinite μ] [MeasureTheory.SigmaFinite ν] (hνμ : ν.AbsolutelyContinuous μ) :
(fun (x : α) => Real.exp (a * MeasureTheory.llr μ ν x)) =ᵐ[ν] fun (x : α) => (μ.rnDeriv ν x).toReal ^ a
theorem ProbabilityTheory.exp_mul_llr' {α : Type u_1} {mα : MeasurableSpace α} {μ : MeasureTheory.Measure α} {ν : MeasureTheory.Measure α} {a : } [MeasureTheory.SigmaFinite μ] [MeasureTheory.SigmaFinite ν] (hμν : μ.AbsolutelyContinuous ν) :
(fun (x : α) => Real.exp (a * MeasureTheory.llr μ ν x)) =ᵐ[μ] fun (x : α) => (μ.rnDeriv ν x).toReal ^ a
noncomputable def ProbabilityTheory.renyiDiv {α : Type u_1} {mα : MeasurableSpace α} (a : ) (μ : MeasureTheory.Measure α) (ν : MeasureTheory.Measure α) :

Rényi divergence of order a. If a = 1, it is defined as kl μ ν, otherwise as (a - 1)⁻¹ * log (ν(α) + (a - 1) * Hₐ(μ, ν)). If ν is a probability measure then this becomes the more usual definition (a - 1)⁻¹ * log (1 + (a - 1) * Hₐ(μ, ν)), but this definition maintains some useful properties also for a general finite measure ν, in particular the integral form Rₐ(μ, ν) = (a - 1)⁻¹ * log (∫ x, ((∂μ/∂ν) x) ^ a ∂ν). We use ENNReal.log instead of Real.log, because it is monotone on ℝ≥0∞, while the real log is monotone only on (0, ∞) (Real.log 0 = 0). This allows us to transfer inequalities from the Hellinger divergence to the Rényi divergence.

Equations
Instances For
    @[simp]
    theorem ProbabilityTheory.renyiDiv_of_ne_one {α : Type u_1} {mα : MeasurableSpace α} {a : } (ha_ne_one : a 1) (μ : MeasureTheory.Measure α) (ν : MeasureTheory.Measure α) :
    ProbabilityTheory.renyiDiv a μ ν = (a - 1)⁻¹ * ((ν Set.univ) + (a - 1) * ProbabilityTheory.hellingerDiv a μ ν).toENNReal.log
    theorem ProbabilityTheory.renyiDiv_eq_top_iff_of_one_lt {α : Type u_1} {mα : MeasurableSpace α} {μ : MeasureTheory.Measure α} {ν : MeasureTheory.Measure α} {a : } (ha : 1 < a) [MeasureTheory.SigmaFinite μ] [MeasureTheory.IsFiniteMeasure ν] :
    ProbabilityTheory.renyiDiv a μ ν = ¬MeasureTheory.Integrable (fun (x : α) => (μ.rnDeriv ν x).toReal ^ a) ν ¬μ.AbsolutelyContinuous ν
    theorem ProbabilityTheory.renyiDiv_ne_top_iff_of_one_lt {α : Type u_1} {mα : MeasurableSpace α} {μ : MeasureTheory.Measure α} {ν : MeasureTheory.Measure α} {a : } (ha : 1 < a) [MeasureTheory.SigmaFinite μ] [MeasureTheory.IsFiniteMeasure ν] :
    ProbabilityTheory.renyiDiv a μ ν MeasureTheory.Integrable (fun (x : α) => (μ.rnDeriv ν x).toReal ^ a) ν μ.AbsolutelyContinuous ν
    theorem ProbabilityTheory.renyiDiv_of_mutuallySingular {α : Type u_1} {mα : MeasurableSpace α} {μ : MeasureTheory.Measure α} {ν : MeasureTheory.Measure α} {a : } (ha_nonneg : 0 a) [NeZero μ] [MeasureTheory.IsFiniteMeasure μ] [MeasureTheory.IsFiniteMeasure ν] (hμν : μ.MutuallySingular ν) :
    theorem ProbabilityTheory.renyiDiv_eq_log_integral_of_lt_one {α : Type u_1} {mα : MeasurableSpace α} {μ : MeasureTheory.Measure α} {ν : MeasureTheory.Measure α} {a : } (ha_pos : 0 < a) (ha : a < 1) [MeasureTheory.IsFiniteMeasure μ] [MeasureTheory.IsFiniteMeasure ν] :
    ProbabilityTheory.renyiDiv a μ ν = (a - 1)⁻¹ * (ENNReal.ofReal (∫ (x : α), (μ.rnDeriv ν x).toReal ^ aν)).log

    The Rényi divergence renyiDiv a μ ν can be written as the log of an integral with respect to ν.

    theorem ProbabilityTheory.renyiDiv_eq_log_integral {α : Type u_1} {mα : MeasurableSpace α} {μ : MeasureTheory.Measure α} {ν : MeasureTheory.Measure α} {a : } (ha_pos : 0 < a) (ha_ne_one : a 1) [MeasureTheory.IsFiniteMeasure μ] [MeasureTheory.IsFiniteMeasure ν] (h_int : MeasureTheory.Integrable (fun (x : α) => (μ.rnDeriv ν x).toReal ^ a) ν) (h_ac : μ.AbsolutelyContinuous ν) :
    ProbabilityTheory.renyiDiv a μ ν = (a - 1)⁻¹ * (ENNReal.ofReal (∫ (x : α), (μ.rnDeriv ν x).toReal ^ aν)).log

    The Rényi divergence renyiDiv a μ ν can be written as the log of an integral with respect to ν. If a < 1, use renyiDiv_eq_log_integral_of_lt_one instead.

    theorem ProbabilityTheory.renyiDiv_eq_log_integral_of_lt_one' {α : Type u_1} {mα : MeasurableSpace α} {μ : MeasureTheory.Measure α} {ν : MeasureTheory.Measure α} {a : } (ha_pos : 0 < a) (ha : a < 1) [MeasureTheory.IsFiniteMeasure μ] [MeasureTheory.IsFiniteMeasure ν] (h_ac : μ.AbsolutelyContinuous ν) :
    ProbabilityTheory.renyiDiv a μ ν = (a - 1)⁻¹ * (ENNReal.ofReal (∫ (x : α), (μ.rnDeriv ν x).toReal ^ (a - 1)μ)).log

    If μ ≪ ν, the Rényi divergence renyiDiv a μ ν can be written as the log of an integral with respect to μ.

    theorem ProbabilityTheory.renyiDiv_eq_log_integral' {α : Type u_1} {mα : MeasurableSpace α} {μ : MeasureTheory.Measure α} {ν : MeasureTheory.Measure α} {a : } (ha_pos : 0 < a) (ha : a 1) [MeasureTheory.IsFiniteMeasure μ] [MeasureTheory.IsFiniteMeasure ν] (h_int : MeasureTheory.Integrable (fun (x : α) => (μ.rnDeriv ν x).toReal ^ a) ν) (h_ac : μ.AbsolutelyContinuous ν) :
    ProbabilityTheory.renyiDiv a μ ν = (a - 1)⁻¹ * (ENNReal.ofReal (∫ (x : α), (μ.rnDeriv ν x).toReal ^ (a - 1)μ)).log

    If μ ≪ ν, the Rényi divergence renyiDiv a μ ν can be written as the log of an integral with respect to μ. If a < 1, use renyiDiv_eq_log_integral_of_lt_one' instead.

    theorem ProbabilityTheory.renyiDiv_symm' {α : Type u_1} {mα : MeasurableSpace α} {μ : MeasureTheory.Measure α} {ν : MeasureTheory.Measure α} {a : } (ha_pos : 0 < a) (ha : a < 1) (h_eq : μ Set.univ = ν Set.univ) [MeasureTheory.IsFiniteMeasure μ] [MeasureTheory.IsFiniteMeasure ν] :
    (1 - a) * ProbabilityTheory.renyiDiv a μ ν = a * ProbabilityTheory.renyiDiv (1 - a) ν μ
    theorem ProbabilityTheory.coe_cgf_llr_of_lt_one {α : Type u_1} {mα : MeasurableSpace α} {μ : MeasureTheory.Measure α} {ν : MeasureTheory.Measure α} {a : } (ha_pos : 0 < a) (ha : a < 1) [hν : NeZero ν] [MeasureTheory.IsFiniteMeasure μ] [MeasureTheory.IsFiniteMeasure ν] (hνμ : ν.AbsolutelyContinuous μ) :
    theorem ProbabilityTheory.cgf_llr_of_lt_one {α : Type u_1} {mα : MeasurableSpace α} {μ : MeasureTheory.Measure α} {ν : MeasureTheory.Measure α} {a : } (ha_pos : 0 < a) (ha : a < 1) [MeasureTheory.IsFiniteMeasure μ] [MeasureTheory.IsFiniteMeasure ν] (hνμ : ν.AbsolutelyContinuous μ) :
    theorem ProbabilityTheory.coe_cgf_llr' {α : Type u_1} {mα : MeasurableSpace α} {μ : MeasureTheory.Measure α} {ν : MeasureTheory.Measure α} {a : } (ha_pos : 0 < a) [hν : NeZero μ] [MeasureTheory.IsFiniteMeasure μ] [MeasureTheory.IsFiniteMeasure ν] (h_int : MeasureTheory.Integrable (fun (x : α) => (μ.rnDeriv ν x).toReal ^ (1 + a)) ν) (hμν : μ.AbsolutelyContinuous ν) :
    theorem ProbabilityTheory.cgf_llr' {α : Type u_1} {mα : MeasurableSpace α} {μ : MeasureTheory.Measure α} {ν : MeasureTheory.Measure α} {a : } (ha_pos : 0 < a) [MeasureTheory.IsFiniteMeasure μ] [MeasureTheory.IsFiniteMeasure ν] (h_int : MeasureTheory.Integrable (fun (x : α) => (μ.rnDeriv ν x).toReal ^ (1 + a)) ν) (hμν : μ.AbsolutelyContinuous ν) :
    noncomputable def ProbabilityTheory.renyiDensity {α : Type u_1} {mα : MeasurableSpace α} (a : ) (μ : MeasureTheory.Measure α) (ν : MeasureTheory.Measure α) (x : α) :

    Density of the Rényi measure renyiMeasure a μ ν with respect to μ + ν.

    Equations
    Instances For

      Tilted measure of μ with respect to ν parametrized by a.

      Equations
      Instances For
        theorem ProbabilityTheory.le_renyiDiv_of_le_hellingerDiv {α : Type u_1} {mα : MeasurableSpace α} {β : Type u_3} {mβ : MeasurableSpace β} {a : } {μ₁ : MeasureTheory.Measure α} {ν₁ : MeasureTheory.Measure α} {μ₂ : MeasureTheory.Measure β} {ν₂ : MeasureTheory.Measure β} [MeasureTheory.SigmaFinite μ₁] [MeasureTheory.SigmaFinite ν₁] [MeasureTheory.SigmaFinite μ₂] [MeasureTheory.SigmaFinite ν₂] (h_eq : ν₁ Set.univ = ν₂ Set.univ) (h_le : ProbabilityTheory.hellingerDiv a μ₁ ν₁ ProbabilityTheory.hellingerDiv a μ₂ ν₂) :

        The Data Processing Inequality for the Renyi divergence.