---
_id: '11839'
abstract:
- lang: eng
  text: "It is a highly desirable property for deep networks to be robust against\r\nsmall
    input changes. One popular way to achieve this property is by designing\r\nnetworks
    with a small Lipschitz constant. In this work, we propose a new\r\ntechnique for
    constructing such Lipschitz networks that has a number of\r\ndesirable properties:
    it can be applied to any linear network layer\r\n(fully-connected or convolutional),
    it provides formal guarantees on the\r\nLipschitz constant, it is easy to implement
    and efficient to run, and it can be\r\ncombined with any training objective and
    optimization method. In fact, our\r\ntechnique is the first one in the literature
    that achieves all of these\r\nproperties simultaneously. Our main contribution
    is a rescaling-based weight\r\nmatrix parametrization that guarantees each network
    layer to have a Lipschitz\r\nconstant of at most 1 and results in the learned
    weight matrices to be close to\r\northogonal. Hence we call such layers almost-orthogonal
    Lipschitz (AOL).\r\nExperiments and ablation studies in the context of image classification
    with\r\ncertified robust accuracy confirm that AOL layers achieve results that
    are on\r\npar with most existing methods. Yet, they are simpler to implement and
    more\r\nbroadly applicable, because they do not require computationally expensive\r\nmatrix
    orthogonalization or inversion steps as part of the network\r\narchitecture. We
    provide code at https://github.com/berndprach/AOL."
alternative_title:
- LNCS
article_processing_charge: No
arxiv: 1
author:
- first_name: Bernd
  full_name: Prach, Bernd
  id: 2D561D42-C427-11E9-89B4-9C1AE6697425
  last_name: Prach
- first_name: Christoph
  full_name: Lampert, Christoph
  id: 40C20FD2-F248-11E8-B48F-1D18A9856A87
  last_name: Lampert
  orcid: 0000-0001-8622-7887
citation:
  ama: 'Prach B, Lampert C. Almost-orthogonal layers for efficient general-purpose
    Lipschitz networks. In: <i>Computer Vision – ECCV 2022</i>. Vol 13681. Springer
    Nature; 2022:350-365. doi:<a href="https://doi.org/10.1007/978-3-031-19803-8_21">10.1007/978-3-031-19803-8_21</a>'
  apa: 'Prach, B., &#38; Lampert, C. (2022). Almost-orthogonal layers for efficient
    general-purpose Lipschitz networks. In <i>Computer Vision – ECCV 2022</i> (Vol.
    13681, pp. 350–365). Tel Aviv, Israel: Springer Nature. <a href="https://doi.org/10.1007/978-3-031-19803-8_21">https://doi.org/10.1007/978-3-031-19803-8_21</a>'
  chicago: Prach, Bernd, and Christoph Lampert. “Almost-Orthogonal Layers for Efficient
    General-Purpose Lipschitz Networks.” In <i>Computer Vision – ECCV 2022</i>, 13681:350–65.
    Springer Nature, 2022. <a href="https://doi.org/10.1007/978-3-031-19803-8_21">https://doi.org/10.1007/978-3-031-19803-8_21</a>.
  ieee: B. Prach and C. Lampert, “Almost-orthogonal layers for efficient general-purpose
    Lipschitz networks,” in <i>Computer Vision – ECCV 2022</i>, Tel Aviv, Israel,
    2022, vol. 13681, pp. 350–365.
  ista: 'Prach B, Lampert C. 2022. Almost-orthogonal layers for efficient general-purpose
    Lipschitz networks. Computer Vision – ECCV 2022. ECCV: European Conference on
    Computer Vision, LNCS, vol. 13681, 350–365.'
  mla: Prach, Bernd, and Christoph Lampert. “Almost-Orthogonal Layers for Efficient
    General-Purpose Lipschitz Networks.” <i>Computer Vision – ECCV 2022</i>, vol.
    13681, Springer Nature, 2022, pp. 350–65, doi:<a href="https://doi.org/10.1007/978-3-031-19803-8_21">10.1007/978-3-031-19803-8_21</a>.
  short: B. Prach, C. Lampert, in:, Computer Vision – ECCV 2022, Springer Nature,
    2022, pp. 350–365.
conference:
  end_date: 2022-10-27
  location: Tel Aviv, Israel
  name: 'ECCV: European Conference on Computer Vision'
  start_date: 2022-10-23
date_created: 2022-08-12T15:09:47Z
date_published: 2022-10-23T00:00:00Z
date_updated: 2023-05-03T08:00:46Z
day: '23'
department:
- _id: GradSch
- _id: ChLa
doi: 10.1007/978-3-031-19803-8_21
external_id:
  arxiv:
  - '2208.03160'
intvolume: '     13681'
language:
- iso: eng
main_file_link:
- open_access: '1'
  url: ' https://doi.org/10.48550/arXiv.2208.03160'
month: '10'
oa: 1
oa_version: Preprint
page: 350-365
publication: Computer Vision – ECCV 2022
publication_identifier:
  eisbn:
  - '9783031198038'
  isbn:
  - '9783031198021'
publication_status: published
publisher: Springer Nature
quality_controlled: '1'
scopus_import: '1'
status: public
title: Almost-orthogonal layers for efficient general-purpose Lipschitz networks
type: conference
user_id: 2DF688A6-F248-11E8-B48F-1D18A9856A87
volume: 13681
year: '2022'
...