Yeison Liscano

Pruebas de hipótesis paso a paso: una diferencia no basta

Wed, 04 Nov 2026 00:00:00 GMT

Palabras clave: pruebas de hipótesis, hipótesis nula, valor p, significancia estadística, potencia, intervalo de confianza.

En una prueba de cupones con 900 clientes por grupo, el abandono fue del 28 % sin cupón y del 22 % con cupón. Desplegar la campaña cuesta dinero: ¿qué evidencia ofrece esa diferencia para hacerlo? Recorreremos una prueba de hipótesis desde su planificación hasta el informe final. Veremos cómo formular la pregunta, elegir el contraste, calcular el valor p y evaluar la importancia práctica del efecto.

Retomamos la tienda de los cuatro tipos de análisis de datos. Los recuentos son un ejemplo didáctico, no datos de una campaña real. Aquí suponemos asignación aleatoria, seguimiento completo y una observación independiente por cliente. El código calcula los resultados de esos recuentos; no demuestra que tales condiciones se cumplan en una tienda real.

La pregunta debe convertirse en una afirmación medible

«El cupón funciona» deja demasiadas cosas abiertas. Nuestra pregunta será: ¿cambia un cupón del 15 % la proporción de clientes que no compra durante los 90 días siguientes? La población objetivo son los clientes elegibles para esa campaña. El resultado de cada cliente es binario: abandonó o no abandonó.

El parámetro que queremos estimar es la diferencia de proporciones:

Δ = proporción de abandono sin cupón − proporción con cupón
H₀: Δ = 0       (no hay diferencia)
H₁: Δ ≠ 0       (hay diferencia en alguna dirección)

La hipótesis nula, H₀, proporciona el escenario de referencia. La alternativa, H₁, expresa qué diferencias queremos detectar. Con nuestra convención, un Δ positivo favorece al cupón.

Elegimos una prueba bilateral porque nos interesa detectar tanto una reducción como un aumento del abandono. Una alternativa unilateral, Δ > 0, respondería otra pregunta. Debe justificarse antes de observar los resultados; no se escoge después para obtener un valor p menor.

La falsabilidad se explica en el método científico aplicado al software y la seguridad. Aquí añadimos una dificultad: las muestras fluctúan. Una diferencia observada no refuta por sí sola una hipótesis estadística.

El diseño y la muestra se deciden antes del resultado

Antes de enviar el primer cupón, dejamos escrito el plan:

Unidad y asignación: cada cliente pertenece a un solo grupo, elegido al azar. Diez compras del mismo cliente no son diez clientes independientes.
Métrica principal: abandono a 90 días, con la misma definición en ambos grupos. Fijamos cómo tratar datos faltantes y exclusiones.
Contraste: bilateral de dos proporciones independientes, con nivel de significancia α = 0,05.
Muestra y parada: 900 clientes por grupo en este ejemplo, todos con su ventana de seguimiento completa. El número es ilustrativo; no resulta aquí de un cálculo de potencia.
Relevancia práctica: qué reducción justificaría el costo y qué otros resultados, como margen o quejas, debemos vigilar.

El nivel de significancia, α, controla un error del procedimiento. Con α = 0,05, una prueba válida tiene una tasa de falsos rechazos de hasta el 5 % cuando H₀ es cierta. No significa que nuestro resultado tenga un 5 % de probabilidad de ser un error.

Hay dos errores posibles:

Decisión	H₀ es cierta	Existe una diferencia
Rechazar H₀	Error de tipo I: falso positivo	Detectamos una diferencia
No rechazar H₀	Decisión correcta: no rechazar H₀	Error de tipo II: falso negativo

La potencia es la probabilidad de detectar un efecto especificado. «Potencia del 80 %» está incompleto si no indicamos para qué diferencia. Planificar la muestra requiere una tasa de abandono esperada, un efecto que queremos detectar, α y la potencia deseada. Bajar α reduce falsos positivos, pero, manteniendo muestra y efecto, también reduce la potencia.

El manual del NIST (National Institute of Standards and Technology (Instituto Nacional de Estándares y Tecnología de Estados Unidos)) explica estas relaciones en su introducción a los contrastes. Ningún cálculo de muestra corrige una asignación sesgada o clientes contados varias veces como si fueran independientes.

La prueba depende de los datos y de cómo se obtuvieron

La elección empieza por el diseño y la variable que medimos. Esta tabla orienta; no sustituye comprobar los supuestos:

Situación	Prueba posible	Condición que debemos revisar
Dos proporciones independientes	z de dos proporciones	Suficientes eventos y no eventos para la aproximación normal
Tabla 2 × 2 con recuentos pequeños	Exacta de Fisher	Diseño compatible con su modelo y condicionamiento a los márgenes
Medias de dos grupos independientes	t de Welch	Independencia; con muestras pequeñas importa la distribución
Mediciones numéricas emparejadas	t de las diferencias	Parejas independientes; con pocas parejas importa la normalidad de las diferencias

La documentación oficial describe la prueba de Fisher y la prueba de Welch. El NIST explica por qué las observaciones emparejadas se analizan como diferencias. Welch permite varianzas distintas, pero no elimina la dependencia ni resuelve automáticamente una asimetría extrema.

Para los cupones elegimos z: hay dos grupos independientes y suficientes abandonos y permanencias en ambos. Suponemos también que el cupón de un cliente no cambia el resultado de otro. La asignación aleatoria ayuda a interpretar el efecto causal en el experimento; generalizarlo a otros clientes exige justificar que el contexto es comparable.

El valor p mide la incompatibilidad con la hipótesis nula

Al cerrar el seguimiento obtenemos estos recuentos:

Grupo	Abandonaron	Permanecieron	Total	Abandono
Sin cupón	252	648	900	28 %
Con cupón	198	702	900	22 %

La diferencia observada es 6 puntos porcentuales. Bajo H₀ suponemos una misma proporción de abandono en ambos grupos. La estimamos agrupando sus recuentos: 450 abandonos entre 1800 clientes, es decir, 0,25.

El estadístico z divide la diferencia por su error estándar bajo H₀:

p_agrupada = (252 + 198) / (900 + 900) = 0,25
error_H₀ = raíz[p_agrupada × (1 − p_agrupada) × (1/900 + 1/900)]
z = (0,28 − 0,22) / error_H₀ ≈ 2,939

La diferencia queda a unas 2,94 unidades de error estándar del cero. Usamos la aproximación normal descrita por NIST.

El valor p bilateral es la probabilidad, bajo H₀ y los demás supuestos, de obtener un estadístico al menos tan extremo como el observado, en cualquiera de las dos direcciones. Para este contraste sumamos las dos colas más allá de −2,939 y +2,939. El resultado es aproximadamente 0,00329.

Figure: Bajo H₀, las dos colas más allá de ±2,939 suman p ≈ 0,00329. El área se refiere al comportamiento del estadístico bajo ese modelo, no a la probabilidad de que H₀ sea cierta. (interactive version in the post)

Este código usa solo la biblioteca estándar de Python. erfc permite calcular las colas normales sin instalar paquetes:

from math import erfc, sqrt

n_control, n_cupon = 900, 900
x_control, x_cupon = 252, 198
p_control = x_control / n_control
p_cupon = x_cupon / n_cupon
delta = p_control - p_cupon

agrupada = (x_control + x_cupon) / (n_control + n_cupon)
error_nulo = sqrt(
    agrupada * (1 - agrupada) * (1 / n_control + 1 / n_cupon)
)
z = delta / error_nulo
p_bilateral = erfc(abs(z) / sqrt(2))

# Intervalo de Wald aproximado: error estimado sin agrupar.
error_estimacion = sqrt(
    p_control * (1 - p_control) / n_control
    + p_cupon * (1 - p_cupon) / n_cupon
)
# Cuantil 0,975 de la normal estándar para un IC bilateral del 95 %.
margen = 1.959963984540054 * error_estimacion
inferior, superior = delta - margen, delta + margen

print(f"Diferencia: {100 * delta:.2f} puntos porcentuales")
print(f"z: {z:.6f}")
print(f"p bilateral: {p_bilateral:.8f}")
print(f"IC 95% aproximado: [{100 * inferior:.3f}, {100 * superior:.3f}] pp")

La ejecución produce:

Diferencia: 6.00 puntos porcentuales
z: 2.939388
p bilateral: 0.00328861
IC 95% aproximado: [2.009, 9.991] pp

Como p < 0,05, rechazamos H₀ con el criterio fijado. La diferencia positiva aporta evidencia de menor abandono con cupón bajo los supuestos del análisis. Eso todavía no decide si conviene desplegarlo.

La significancia debe acompañarse del efecto y su incertidumbre

El valor p no es la probabilidad de que H₀ sea cierta. Tampoco mide el tamaño del efecto ni su importancia económica. Son distinciones centrales de la declaración de la ASA (American Statistical Association (Asociación Estadística Estadounidense)) sobre significancia estadística y valores p.

El efecto estimado es una reducción de 6 puntos porcentuales. Su IC (Intervalo de confianza) del 95 % va de 2,01 a 9,99 puntos. En muestras repetidas, el procedimiento produce intervalos que cubren el parámetro en aproximadamente el 95 % de los casos, bajo sus supuestos. El intervalo observado no asigna una probabilidad del 95 % a un parámetro fijo.

Usamos un intervalo de Wald aproximado, con el error sin agrupar. La documentación de statsmodels para dos proporciones incluye este método y otras alternativas. No debe aplicarse automáticamente con pocos eventos o proporciones cercanas a cero o uno. Además, este intervalo y el contraste agrupado no son inversiones exactas del mismo procedimiento.

Ahora supongamos que el negocio hubiera fijado antes un requisito: reducir el abandono más de 5 puntos para justificar el costo. El intervalo incluye valores por debajo de ese umbral. Encontramos evidencia frente a cero, pero no hemos establecido que el efecto supere 5 puntos.

Figure: El intervalo aproximado excluye cero, pero contiene el umbral práctico ilustrativo de 5 puntos. Una diferencia estadísticamente significativa puede dejar sin resolver la decisión económica. (interactive version in the post)

Para evaluar directamente ese requisito habría que planificar un contraste frente al umbral, con su potencia correspondiente, y considerar costos y márgenes. El umbral no debe aparecer solo porque el resultado nos gustó.

El caso contrario también importa: no rechazar H₀ no demuestra igualdad. Una muestra puede ser demasiado pequeña para detectar una diferencia relevante. Si queremos demostrar que dos opciones son suficientemente parecidas, necesitamos una prueba de equivalencia con márgenes definidos previamente.

Buscar significancia hasta encontrarla altera la prueba

En el diseño de muestra fija, mirar cada día y detenerse cuando p baja de 0,05 modifica la tasa de falsos positivos. También la modifica probar muchas métricas o segmentos y publicar solo el resultado favorable.

Un estudio de Simmons, Nelson y Simonsohn documenta estos riesgos. La defensa práctica es fijar métrica, exclusiones, análisis y parada antes de observar los resultados. Si hay varias hipótesis confirmatorias, hay que planificar cómo controlar los errores de ese conjunto.

Se puede detener un experimento antes mediante un diseño secuencial adecuado. Lo que cambia es el procedimiento y sus reglas, no solo el momento de mirar. Podemos explorar segmentos después, pero debemos presentar sus hallazgos como exploratorios y buscar confirmación con datos nuevos.

La conclusión debe responder la pregunta y declarar sus límites

El informe del ejemplo puede quedar así:

En este caso didáctico, el abandono a 90 días fue del 28 % sin cupón y del 22 % con cupón, con 900 clientes por grupo. La reducción estimada fue de 6 puntos porcentuales, con IC de Wald aproximado del 95 % de 2,01 a 9,99. El contraste bilateral de dos proporciones dio p = 0,00329. Con α = 0,05, rechazamos la igualdad bajo los supuestos declarados. El resultado aporta evidencia de una reducción, pero no establece que supere 5 puntos ni que la campaña sea rentable.

Antes de la próxima prueba, escribe una frase con qué vas a medir, qué efecto importa y cuándo terminarás de observar. Esa frase hace que el cálculo responda una pregunta definida, en lugar de dejar que el resultado elija la pregunta.

Referencias

El NIST/SEMATECH e-Handbook: Quantitative Techniques explica los fundamentos del contraste.
La documentación de NIST Dataplot: Difference of Proportion Hypothesis Test presenta el estadístico del ejemplo.
La documentación de SciPy: fisher_exact describe una alternativa exacta para tablas 2 × 2.
La documentación de SciPy: ttest_ind explica la comparación de medias independientes.
El manual de NIST: Analysis of paired observations desarrolla el caso emparejado.
La declaración de la ASA sobre valores p (7 de marzo de 2016) fundamenta sus límites de interpretación.
La documentación de statsmodels: confint_proportions_2indep detalla los métodos de intervalos disponibles.
El artículo de Simmons, Nelson y Simonsohn, False-Positive Psychology (Psychological Science, 2011) examina el efecto de las decisiones de análisis no declaradas.

5 Pourquoi : le cas d'école du Jefferson Memorial n'a pas la fin qu'on raconte, et comment chercher plus d'une cause

Tue, 03 Nov 2026 00:00:00 GMT

Mots-clés : 5 Pourquoi, analyse des causes racines, arbre des causes, INRS, postmortem, erreur humaine, Jefferson Memorial, Toyota

Dans les formations aux 5 Pourquoi, un exemple revient sans cesse. Les pierres du Jefferson Memorial s'abîment. Pourquoi ? On les lave trop souvent. Pourquoi ? À cause des fientes d'oiseaux, attirés par des araignées, elles-mêmes attirées par des moucherons, qui viennent pour les projecteurs. On allume donc les lumières une heure plus tard, et le problème disparaît. Les archives racontent autre chose. L'essai a surtout eu lieu au Lincoln Memorial, l'eau du nettoyage menaçait davantage la pierre, et l'éclairage retardé a été abandonné après les plaintes des touristes.

Une seule chaîne de pourquoi avait caché plusieurs causes et une contrainte. L'enjeu dépasse les monuments : dans un grand centre hospitalier, 302 analyses de causes ont surtout produit des actions faibles, et plusieurs types d'événements se sont répétés. Cet article reprend le cas, puis le compare à deux postmortems réels, celui de GitLab et celui d'AWS, et à l'arbre des causes de l'INRS. Vous repartirez avec des règles pour savoir quand s'arrêter, et un script qui signale les pourquoi qui s'arrêtent sur une personne ou sur un « manque ».

Le cas d'école tient en une chaîne, une action et un chiffre

En septembre 2018, un équipementier automobile, Nexteer, publie un support de formation aux 5 Pourquoi pour ses fournisseurs. En annexe, une diapositive intitulée « A good example » raconte l'histoire du Jefferson Memorial. Le National Park Service remarque que le monument se dégrade plus vite que les autres. Les gardes mènent un 5 Pourquoi et obtiennent cette chaîne :

Le mémorial se dégrade plus vite, parce qu'il est lavé plus souvent.
Il est lavé plus souvent, parce qu'il reçoit plus de fientes.
Il reçoit plus de fientes, parce que plus d'oiseaux viennent.
Plus d'oiseaux viennent, parce que plus d'araignées vivent autour.
Plus d'araignées vivent autour, parce que plus d'insectes volent au crépuscule.
Plus d'insectes volent au crépuscule, parce que l'éclairage les attire.

La diapositive se demande ensuite s'il faut aller plus loin. Pourquoi la lumière attire-t-elle les insectes ? La réponse ne serait pas « within their control », donc on s'arrête. Elle envisage aussi de s'arrêter plus tôt : enduire la pierre d'un produit hydrofuge pour supporter les lavages, ou traiter les araignées aux pesticides. C'est possible, juge-t-elle, mais d'autres causes restent à considérer. Les gardes retiennent l'éclairage. Action : allumer une heure plus tard. Résultat annoncé : « Bird dropping problem reduced by 90%! »

L'histoire plaît parce que chaque réponse appelle la suivante, que l'action est gratuite et que le résultat est un chiffre. Elle compte d'ailleurs six pourquoi, pas cinq, ce qui n'a pas d'importance : le lexique du Lean Enterprise Institute, qui cite Taiichi Ohno, précise que « the specific number five is not the point ». L'exemple de la machine d'Ohno est raconté dans l'article sur le 5W2H. Mais trois questions restent ouvertes. Qui a mesuré ces 90 % ? Qu'a-t-on essayé d'autre ? Et pourquoi une seule cause à chaque maillon ?

Les archives montrent d'autres causes, un autre mémorial, un essai abandonné

En avril 1990, une dépêche AP (Associated Press) reprise par le Deseret News annonce une étude à 2 millions de dollars, commandée par le National Park Service à des consultants privés. Elle porte sur le vieillissement des deux mémoriaux, Lincoln et Jefferson, et recense plusieurs menaces. Le moucheron n'est que l'une d'elles.

En mai 1992, Carla Helfferich raconte la suite dans une chronique scientifique de l'université d'Alaska à Fairbanks. Le monument est cette fois le Lincoln Memorial. Les projecteurs attirent des nuées de moucherons au crépuscule. Les araignées s'en nourrissent, et les oiseaux mangent les araignées. Les équipes de nettoyage doivent laver fientes, toiles et moucherons écrasés, et l'eau du lavage fait entrer dans le marbre les polluants de l'air. Le Park Service laisse alors les projecteurs éteints une heure de plus. Le nombre de moucherons morts sur la statue baisse de 90 %.

Joel Gross a reconstitué toute l'affaire en 2014, dans une enquête qu'Alan Card cite dans sa critique des 5 Pourquoi. Selon Gross, le rapport des consultants désigne une menace principale : le grand volume d'eau utilisé au nettoyage. L'eau pénètre dans les inclusions du marbre et du calcaire, et fait progresser les fissures.

L'essai d'éclairage dure six semaines, les lundis, mercredis et vendredis, avec 85 % de moucherons en moins. En janvier 1992, le Park Service étudie encore un changement permanent. En septembre 1995, l'idée est abandonnée, à cause du « nombre écrasant de plaintes de touristes venus prendre des photos ».

Le même article de 1995 décrit les autres mesures prises en cinq ans : des fils, des pics, des filets et du plastique transparent, contre les oiseaux, les araignées et les moucherons. Selon Gross, elles ont peu réduit le nettoyage.

Les sources ne s'accordent ni sur l'année de l'essai ni sur le chiffre : 90 % selon la chronique de 1992, 85 % selon Gross. Elles s'accordent sur l'essentiel, résumé dans ce tableau :

Point	Version racontée	Archives
Lieu	Jefferson Memorial	Surtout le Lincoln Memorial
Menace	Lavages fréquents	L'eau du lavage qui pénètre la pierre fissurée
Ce qui est mesuré	Fientes, −90 %	Moucherons morts, −85 % ou −90 %
Statut	Solution	Essai de six semaines
Suite	Problème réglé	Abandonné en 1995, après les plaintes des touristes

Personne n'a mesuré une baisse de 90 % des fientes. Et l'action retenue n'a jamais été appliquée durablement.

Une chaîne de pourquoi choisit une branche et cache les autres

La chaîne racontée n'est pas fausse : chacun de ses maillons est plausible, et l'essai a bien réduit les moucherons. Son défaut est ce qu'elle laisse de côté. À chaque pourquoi, elle ne retient qu'une réponse. Or la pierre ne s'abîme pas seulement parce qu'on la lave : il faut aussi que l'eau entre dans le marbre, et qu'elle y apporte des polluants. La figure compare les deux lectures, étape par étape.

Figure: La chaîne racontée suit la colonne principale de l'arbre. Les archives ajoutent deux conjonctions, une contrainte qui bloque l'action choisie, et un autre levier essayé en parallèle. (interactive version in the post)

Ce que la chaîne ne voit pas a deux origines. D'abord, elle cherche une cause et s'arrête dès qu'elle en tient une. La fiche de l'IHI sur les 5 Pourquoi rappelle pourtant qu'un problème peut avoir plusieurs causes racines. Ensuite, elle se construit en salle, par déduction. Teruyuki Minoura, ancien dirigeant de Toyota, cité par TapRooT d'après une page de Toyota aujourd'hui retirée, le dit sans détour. Ohno demandait de poser « pourquoi ? » cinq fois. Mais seule l'observation sur le terrain permet de répondre, et les équipes « reel off five causes as quick as a flash by deduction ». Les consultants des mémoriaux, eux, avaient examiné la pierre. C'est ainsi qu'ils ont trouvé l'eau.

Même l'exemple d'Ohno n'échappe pas à la critique. Mark Graban montre en 2026, sur le Lean Blog, que la chaîne de la machine compte en réalité sept pourquoi, et qu'elle s'arrête trop tôt : pourquoi n'y avait-il pas de filtre ?

GitLab et AWS ont corrigé les conditions, pas la personne

Le 31 janvier 2017, un ingénieur de GitLab supprime le répertoire de données de la base principale, au lieu de celui de la base secondaire. GitLab.com reste indisponible environ 18 heures. Le postmortem, publié le 10 février, contient une section intitulée « The 5 Whys ». Elle traite deux problèmes, et donc deux chaînes.

La première explique la panne en cinq pourquoi. Le répertoire a été supprimé par accident. Il fallait le vider parce que la réplication s'était arrêtée et que reconstruire la base secondaire exigeait un répertoire vide, une étape ni automatisée ni bien documentée. La réplication s'était arrêtée à cause d'un pic de charge. Le pic venait de deux choses à la fois : une hausse du spam, et un traitement qui supprimait un employé de GitLab et ses données. L'employé avait été signalé pour abus par un troll, dans un système où il était trop facile d'ignorer les détails d'un signalement.

La seconde chaîne explique pourquoi la restauration a pris plus de 18 heures, en huit pourquoi. Les sauvegardes pg_dump échouaient sans bruit : elles utilisaient PostgreSQL 9.2 contre une base en 9.6. Les courriels d'échec étaient rejetés faute de signature DMARC (Domain-based Message Authentication, Reporting and Conformance). Le dernier pourquoi aboutit à une cause organisationnelle : la procédure de sauvegarde n'avait pas de responsable, donc personne ne la testait.

GitLab ne s'arrête pas à l'erreur de l'ingénieur, qui n'est que le premier pourquoi. Et ses actions visent chacune une cause de la chaîne :

Pourquoi du postmortem	Action de GitLab
Répertoire du primaire supprimé par accident	Invite de commande plus distincte d'un hôte à l'autre
Reconstruction du secondaire ni automatisée ni documentée	Documentation et runbooks de réplication
Sauvegardes `pg_dump` en échec silencieux	Surveillance des sauvegardes avec Prometheus
Procédure de sauvegarde jamais testée, sans responsable	Tests automatiques de restauration, un responsable désigné

Un mois plus tard, le 28 février 2017, le service S3 d'AWS tombe dans la région us-east-1. Le résumé d'AWS décrit le déclencheur sans nommer de coupable : « an authorized S3 team member using an established playbook executed a command ». Il ajoute : « one of the inputs to the command was entered incorrectly ». AWS n'utilise pas les 5 Pourquoi. Mais ses correctifs portent sur les conditions, pas sur la personne : l'outil retire désormais la capacité plus lentement, et refuse de descendre sous le minimum requis par chaque sous-système. AWS ajoute : « This will prevent an incorrect input from triggering a similar event in the future », et vérifie ses autres outils. Une chaîne qui s'arrêterait à « une entrée mal saisie » n'aurait produit aucune de ces actions.

S'arrêter sur une personne ou sur un manque ferme l'analyse

Deux règles reviennent dans les méthodes sérieuses. La première interdit de s'arrêter sur une personne. Le centre de sécurité des patients des anciens combattants américains (VA NCPS (Veterans Affairs National Center for Patient Safety)) a formulé cinq règles de causalité. Un support de formation du New Jersey les reprend, dont celle-ci : toute erreur humaine doit avoir une cause antérieure. Le Betsy Lehman Center précise que ces règles viennent d'un rapport de David Marx pour la FAA (Federal Aviation Administration), en 1999. GitLab respecte cette règle ; une chaîne arrêtée à la commande mal saisie d'AWS la violerait.

La seconde interdit les causes formulées comme un manque. La règle 2 du VA exclut les descriptions négatives des énoncés de cause. L'INRS dit la même chose en français. Dans son guide ED 6163, un fait n'est « ni une interprétation, ni une opinion, ni un jugement de valeur, ni un « fait négatif » (absence de…, manque de…) ». Un fait négatif décrit ce qu'il aurait fallu faire : « Il s'agit donc de bien différencier la cause du remède. » Le guide donne des exemples. « Il ne faisait pas attention » devient « Il parlait avec un collègue ». « Il roulait trop vite » devient « Il roulait à 30 km/h ».

Le postmortem de GitLab contient justement un fait négatif : une étape « ni automatisée ni documentée ». La formule désigne déjà le remède, automatiser et documenter, au lieu du fait : un ingénieur reconstruisait la base secondaire à la main. Le remède est peut-être le bon, mais la formulation empêche d'en envisager un autre.

Derrière ces deux règles, il y a une idée plus large. Richard Cook l'écrit en titre de la septième thèse de How Complex Systems Fail (1998) : « Post-accident attribution to a 'root cause' is fundamentally wrong. » Une panne demande plusieurs défaillances à la fois. Sidney Dekker, cité par John Allspaw, le résume ainsi : « Cause is something we construct, not find. »

L'arbre des causes remplace « pourquoi ? » par « qu'a-t-il fallu ? »

En France, l'INRS (Institut national de recherche et de sécurité) promeut depuis les années soixante-dix une méthode d'analyse des accidents du travail qui ne pose jamais la question « pourquoi ? ». Son guide part d'un constat : « Les accidents du travail ne résultent jamais d'une cause unique : ils sont la conséquence d'une combinaison de facteurs. » L'analyse recherche « une compréhension objective du processus de l'accident et non la recherche de responsabilités ».

L'arbre se construit à partir du dommage, avec deux questions pour chaque fait :

« Qu'a-t-il fallu pour que le fait (Y) apparaisse ? »
« (X) a-t-il été nécessaire à lui seul pour que (Y) apparaisse ? Ou, en d'autres termes, a-t-il fallu autre chose ? »

La seconde question est celle qui manque à la chaîne du mémorial. Elle transforme chaque maillon en nœud, relié à ses causes par un enchaînement, une conjonction ou une disjonction. Le guide en tire une conséquence pratique : « il suffit de supprimer un seul fait pour que l'accident ne se reproduise pas ». Un arbre offre donc plusieurs leviers, et « aucune proposition ne doit être rejetée a priori ». Au mémorial, l'arbre aurait mis côte à côte l'éclairage, le lavage et les oiseaux, avant que les touristes ne tranchent.

John Allspaw arrive au même endroit par l'informatique. Dans The Infinite Hows (2014), il observe que « pourquoi ? » mène trop facilement à « qui ? ». Il propose de demander « comment ? », parce que cette question décrit « (at least some) of the conditions that allowed an event to take place ». « Qu'a-t-il fallu ? » et « comment ? » cherchent des conditions, pas un coupable ni un motif.

Un script repère les pourquoi qui s'arrêtent trop tôt

Ces règles sont assez mécaniques pour qu'un script en vérifie une partie. Le programme ci-dessous représente une analyse comme un arbre : chaque fait pointe vers ses causes. Il signale quatre choses :

les maillons qui n'ont qu'une seule cause, où il faut demander « a-t-il fallu autre chose ? » ;
les faits négatifs (« pas », « ni », « aucun », « absence », « manque »…) ;
les jugements (« trop », « mal », « mauvais », « insuffisant ») ;
les erreurs humaines sans cause antérieure, selon la règle 3 du VA.

Il l'applique à quatre analyses : la chaîne racontée du mémorial, une chaîne naïve sur la panne de S3, construite pour l'exemple, la première chaîne de GitLab, et l'arbre documenté du mémorial. Le bouton Run l'exécute dans votre navigateur.

import re

NEGATIF = re.compile(r"\b(pas|ni|aucun|aucune|absence|manque|sans|jamais)\b")
HUMAIN = re.compile(r"par accident|erreur humaine|inattention|négligence|oubli")
JUGEMENT = re.compile(r"\b(trop|mal|mauvais|insuffisant)\b")


def verifier(nom, arbre):
    """Vérifie une analyse : chaque clé est un fait, sa valeur ses causes."""
    faits = set(arbre) | {c for causes in arbre.values() for c in causes}
    maillons = [fait for fait in faits if arbre.get(fait)]
    uniques = [fait for fait in maillons if len(arbre[fait]) == 1]
    print(f"{nom} : {len(faits)} faits")
    print(f"  ! une seule cause : {len(uniques)} maillons sur {len(maillons)}")
    for fait in sorted(faits):
        if NEGATIF.search(fait):
            print(f"  ! fait négatif : « {fait} »")
        if JUGEMENT.search(fait):
            print(f"  ! jugement : « {fait} »")
        if HUMAIN.search(fait) and not arbre.get(fait):
            print(f"  ! s'arrête sur une erreur humaine : « {fait} »")


def chaine(*faits):
    """Transforme une chaîne de pourquoi en arbre à une seule branche."""
    return {effet: [cause] for effet, cause in zip(faits, faits[1:])}


verifier("Mémorial, version racontée", chaine(
    "Le mémorial se dégrade plus vite",
    "Il est lavé plus souvent",
    "Il reçoit plus de fientes",
    "Plus d'oiseaux viennent",
    "Plus d'araignées vivent autour",
    "Plus d'insectes volent au crépuscule",
    "L'éclairage attire les insectes",
))

verifier("S3, chaîne naïve", chaine(
    "S3 indisponible dans us-east-1",
    "Plus de serveurs retirés que prévu",
    "Une entrée mal saisie, par erreur humaine",
))

verifier("GitLab, problème 1", {
    "GitLab.com indisponible environ 18 heures": [
        "Répertoire du primaire supprimé par accident"],
    "Répertoire du primaire supprimé par accident": [
        "Secondaire à reconstruire, étape ni automatisée ni documentée"],
    "Secondaire à reconstruire, étape ni automatisée ni documentée": [
        "Réplication arrêtée par un pic de charge"],
    "Réplication arrêtée par un pic de charge": [
        "Hausse du spam", "Suppression en cours d'un employé"],
    "Suppression en cours d'un employé": [
        "Signalement d'un troll, détails trop faciles à ignorer"],
})

verifier("Mémorial, arbre documenté", {
    "Pierre qui s'érode": ["Eau infiltrée dans le marbre", "Polluants de l'air"],
    "Eau infiltrée dans le marbre": ["Lavages à grande eau", "Fissures du marbre"],
    "Lavages à grande eau": ["Fientes et insectes à nettoyer"],
    "Fientes et insectes à nettoyer": ["Oiseaux et araignées"],
    "Oiseaux et araignées": ["Moucherons au crépuscule"],
    "Moucherons au crépuscule": ["Projecteurs allumés au crépuscule"],
})

Voici sa sortie, avec Python 3.14 :

Mémorial, version racontée : 7 faits
  ! une seule cause : 6 maillons sur 6
S3, chaîne naïve : 3 faits
  ! une seule cause : 2 maillons sur 2
  ! jugement : « Une entrée mal saisie, par erreur humaine »
  ! s'arrête sur une erreur humaine : « Une entrée mal saisie, par erreur humaine »
GitLab, problème 1 : 7 faits
  ! une seule cause : 4 maillons sur 5
  ! fait négatif : « Secondaire à reconstruire, étape ni automatisée ni documentée »
  ! jugement : « Signalement d'un troll, détails trop faciles à ignorer »
Mémorial, arbre documenté : 9 faits
  ! une seule cause : 4 maillons sur 6

La chaîne racontée du mémorial passe toutes les règles de forme. Elle n'a ni manque, ni jugement, ni coupable. Son seul défaut est structurel : six maillons sur six n'ont qu'une cause. C'est pourquoi elle semble si propre. La chaîne naïve de S3 cumule les défauts : elle juge (« mal saisie ») et s'arrête sur l'erreur humaine. GitLab, lui, continue après l'accident, et l'un de ses maillons a deux causes. Il garde un fait négatif et un jugement, que la reformulation de l'INRS corrigerait.

Le script vérifie la forme, pas la vérité. Une analyse peut passer toutes les règles et rester fausse. Ses expressions régulières se trompent aussi dans les deux sens. Elles signalent « le pas de vis est usé », où « pas » est un nom, et laissent passer « défaut de formation », un manque formulé sans négation. Le script dit où regarder, pas quoi conclure.

Une analyse se juge à la force de ses actions, pas au nombre de pourquoi

Une analyse de causes sert à choisir des actions. C'est donc sur ces actions qu'il faut la juger. En 2017, Kellogg et ses coauteurs ont examiné 302 analyses de causes, menées en huit ans dans un grand centre hospitalier universitaire. Leur étude est parue dans BMJ Quality & Safety. Sur 731 solutions proposées, les plus fréquentes étaient la formation (20 %), un changement de processus (19,6 %) et le rappel d'une règle (15,2 %). Ce sont des actions faibles, et plusieurs types d'événements se sont répétés malgré des analyses successives. La hiérarchie des actions de RCA² (Root Cause Analysis and Action (analyse des causes racines et actions)), présentée dans l'article sur le 5W2H, classe ces actions parmi les plus faibles.

GitLab et AWS ont choisi des actions plus fortes : un outil qui refuse les retraits dangereux, des tests automatiques de restauration, un responsable nommé. Au mémorial, l'éclairage retardé était une action forte, puisqu'elle supprimait une condition. Elle a échoué sur une contrainte que l'analyse n'avait pas cherchée. Les fils et les filets, qui laissaient la condition en place, ont peu changé le nettoyage.

De ces cas et de ces sources, on peut tirer des règles d'arrêt :

Le nombre ne compte pas. On s'arrête quand on atteint un fait sur lequel agir, que ce soit au troisième pourquoi ou au huitième.
Écrire des faits. Pas de manque, pas de jugement : « il parlait avec un collègue », pas « il ne faisait pas attention ».
Ne jamais s'arrêter sur une personne. Toute erreur humaine a une cause antérieure, comme la commande de GitLab ou l'outil d'AWS.
Demander à chaque maillon : a-t-il fallu autre chose ? C'est ce qui transforme la chaîne en arbre.
Vérifier chaque réponse sur place. Les consultants des mémoriaux ont trouvé l'eau en examinant la pierre, pas en déduisant.
Chercher les contraintes avant de choisir. Une action qui supprime une cause peut se heurter à un autre objectif, comme les photos des touristes.

Au mémorial, il manquait une question, pas un pourquoi de plus

Le cas d'école n'a pas la fin qu'on raconte. Les moucherons ont bien diminué, pendant un essai de quelques semaines mené surtout au Lincoln Memorial, puis la solution a été abandonnée. La pierre, elle, souffrait surtout de l'eau. Chaque pourquoi de la chaîne était juste. Ce qui manquait, c'est la question suivante, « a-t-il fallu autre chose ? », qui aurait montré l'eau, les fissures, les polluants et les touristes avant de choisir l'éclairage.

À retenir, et à appliquer au prochain postmortem : à chaque pourquoi, demandez aussi « a-t-il fallu autre chose ? », et n'arrêtez jamais l'analyse sur une personne ou sur un manque. Chaque cause retenue devient ensuite une ligne d'un plan d'action 5W2H, qui la cite dans sa colonne « pourquoi ».

Références

Le cas du mémorial : Nexteer, Supplier 3L5Y Training (2018) ; Associated Press, « Toxic effects of nature are aging Lincoln and Jefferson memorials », Deseret News, 1990 ; C. Helfferich, « Unfortunately ecological Abe », Alaska Science Forum, 1992 ; J. Gross, « 5 Whys Folklore: The Truth Behind a Monumental Mystery », The KaiZone, 2014 (source secondaire, qui cite le New York Times de janvier 1992 et le Chicago Tribune de septembre 1995).
La méthode et ses critiques : Lean Enterprise Institute, 5 Whys, d'après T. Ohno, Toyota Production System, Productivity Press, 1988, p. 17 ; IHI, 5 Whys: Finding the Root Cause ; M. Paradies, « Teruyuki Minoura (Toyota exec) talks about problems with 5 Whys », TapRooT, 2007 (source secondaire) ; M. Graban, « Ohno's 5 Whys Example Is Actually Seven Whys », Lean Blog, 2026 ; A. J. Card, « The problem with '5 whys' », BMJ Quality & Safety, 2017.
Postmortems : GitLab, « Postmortem of database outage of January 31 », 2017 ; Amazon Web Services, « Summary of the Amazon S3 Service Disruption in the Northern Virginia (US-EAST-1) Region », 2017.
Causalité et erreur humaine : New Jersey Department of Health, RCA training (2008), qui reprend les règles du VA NCPS ; Betsy Lehman Center, « The 5 rules of causation » ; R. I. Cook, How Complex Systems Fail (1998) ; J. Allspaw, « The Infinite Hows (or, the Dangers Of The Five Whys) », 2014, qui cite S. Dekker.
Arbre des causes : INRS, La méthode de l'arbre des causes. L'analyse de l'accident du travail, ED 6163, 3e édition, 2025.
Force des actions : K. M. Kellogg et al., « Our current approach to root cause analysis: is it contributing to our failure to improve patient safety? », BMJ Quality & Safety, 2017.

PDCA et IA : vérifier les gains avant de généraliser

Tue, 03 Nov 2026 00:00:00 GMT

Mots-clés : PDCA, intelligence artificielle, amélioration continue, diagnostic, mesure de référence, pilote, qualité de service

Un assistant d’IA (Intelligence artificielle) peut accélérer la rédaction des réponses tout en augmentant les erreurs ou le temps de relecture. Pour une équipe de support, le résultat se mesure sur l’ensemble du traitement d’une demande. À partir d’un pilote fictif, nous suivrons le cycle PDCA (Plan, Do, Check, Act : planifier, faire, vérifier, agir) pour établir une référence, tester une hypothèse, répartir les responsabilités et décider si les résultats justifient un déploiement plus large.

Le PDCA transforme une amélioration supposée en hypothèse à tester

Notre équipe veut réduire le temps consacré aux demandes de support. Elle envisage un assistant qui prépare des réponses à partir de sa documentation. Le projet commence par une question : cette assistance permettra-t-elle de traiter les demandes plus vite, tout en maintenant la qualité ?

Le PDCA donne une structure à cette investigation. La fiche de l’ASQ (American Society for Quality, association professionnelle de la qualité) décrit quatre étapes : planifier un changement, le tester à petite échelle, analyser les résultats, puis agir selon ce qu’on a appris. Si l’essai réussit, on peut étendre le changement et préparer de nouvelles améliorations. Sinon, on révise le plan. Voir la définition du PDCA.

Le cycle aide à organiser l’apprentissage. Il ne fournit ni les indicateurs adaptés à chaque projet ni une preuve automatique d’efficacité. L’ASQ précise d’ailleurs qu’il faut choisir séparément la manière d’analyser les données.

Une nuance historique compte : le Deming Institute rappelle que Deming privilégiait le PDSA (Plan, Do, Study, Act : planifier, faire, étudier, agir). Le mot « étudier » insiste sur la comparaison entre prédiction et observation, puis sur la révision de l’explication. Cette exigence est utile pour notre pilote : nous devons écrire ce qui nous ferait changer d’avis.

Planifier commence par un diagnostic et une mesure de référence

Avant de choisir un modèle, l’équipe observe le parcours d’une demande. Où le temps passe-t-il ? Dans la recherche d’information, la rédaction, l’attente d’un autre service ou la correction d’une première réponse ? Une réponse rédigée plus vite ne raccourcira pas nécessairement une attente de validation.

Le billet sur les 5 Pourquoi explique comment explorer plusieurs causes et vérifier les faits. Ici, nous retenons une hypothèse de travail : la recherche dans la documentation et la rédaction occupent une part du temps sur laquelle l’équipe peut agir.

La mesure de référence, ou baseline, décrit ce fonctionnement avant le changement. Il faut conserver les définitions, les dates et les catégories de demandes. Dans notre exemple, le périmètre couvre uniquement les demandes simples sur l’utilisation du produit. Les litiges et les opérations sensibles restent hors du pilote.

L’équipe formule alors son hypothèse : l’assistance réduira d’au moins 20 % le temps humain moyen par demande, sans augmenter les réouvertures à sept jours. Elle fixe aussi un plafond de coût par demande. Ces seuils sont des choix illustratifs du cas, à adapter au service réel.

Indicateur	Définition conservée pendant l’essai	Référence fictive	Cible du pilote
Temps humain	Temps cumulé de recherche, rédaction, relecture et reprises, par demande	12 min	Au plus 9,6 min
Réouvertures	Part des demandes closes puis rouvertes dans les sept jours	4 %	Au plus 4 %
Coût complet	Travail humain et coûts alloués de l’outil et de son exploitation, par demande	2,40 €	Au plus 2,40 €

La mesure du temps inclut les reprises pendant la fenêtre de suivi. Il faut donc attendre cette fenêtre pour finaliser le bilan. Le taux de réouverture reste imparfait : un client peut abandonner malgré une mauvaise réponse. Une revue d’un échantillon de conversations et les retours des agents complètent ce signal.

Une fois l’hypothèse écrite, la grille QQOQCCP (Qui, quoi, où, quand, comment, combien, pourquoi) aide à transformer le pilote en actions exécutables : qui prépare les données, où l’essai se déroule, quand il se termine et avec quelles ressources. Le billet sur le QQOQCCP en détaille les deux usages.

Faire exige de repenser le processus et les responsabilités

Dans l’ancien fonctionnement, l’agent cherche une page, rédige une réponse et l’envoie. Avec l’assistant, le parcours proposé devient : sélectionner les informations autorisées, produire un brouillon avec ses sources, vérifier ces sources, corriger la réponse, puis l’envoyer. Une demande hors périmètre rejoint le traitement habituel.

C’est le sens opérationnel que nous donnons ici à l’approche « AI First » : repenser le travail à partir des capacités et des limites de l’IA. Cette approche est un choix de conception, pas une exigence du PDCA. Le diagnostic peut aussi conduire à améliorer la documentation ou la recherche sans IA.

L’équipe commence sur un petit périmètre, après des essais hors production. Elle forme les agents à vérifier les réponses et leur laisse la possibilité de refuser une suggestion. Elle définit les responsabilités :

Le responsable du support possède les critères de qualité et décide de l’extension du pilote.
L’équipe technique gère l’intégration, les versions et le retour au fonctionnement précédent.
Les agents vérifient les réponses avant envoi et signalent les erreurs.
Une personne désignée coordonne l’analyse des résultats et le suivi des incidents.

Le cadre de gestion des risques de l’IA du NIST (National Institute of Standards and Technology, institut américain de normalisation et de technologie) recommande de documenter les rôles et les interactions entre personnes et systèmes. Il invite aussi à évaluer si une solution d’IA est appropriée. Nous nous appuyons sur son AI Risk Management Framework 1.0, publié en 2023 et en cours de révision au moment de la rédaction. Ses quatre fonctions ne sont pas des étapes ordonnées du PDCA.

Notre pilote conserve les versions du modèle, des consignes et de la documentation. L’équipe enregistre les corrections avec les précautions adaptées aux données des clients. Elle prévoit aussi comment désactiver l’assistance sans interrompre le support. Ce sont nos choix pour rendre l’essai interprétable et réversible.

Le schéma relie chaque phase à une trace concrète. La décision de la dernière case dépend de l’ensemble des critères fixés dans la première.

Figure: Dans ce cas simulé, le temps baisse de 12 à 9 minutes, mais les réouvertures passent de 4 à 7 % : l’équipe corrige le pilote avant de l’étendre. (interactive version in the post)

Vérifier mesure le service rendu et les effets sur chaque groupe

À la fin de l’essai, l’équipe veut savoir si le changement tient sa promesse. Le nombre de brouillons acceptés renseigne sur l’utilisation de l’outil. Il ne suffit pas à mesurer la résolution des demandes.

Voici un bilan entièrement simulé. Chaque colonne représente 1 000 demandes du périmètre retenu, suivies pendant sept jours après leur clôture. Ces données sont construites pour illustrer une décision ; elles ne proviennent pas d’une entreprise.

Indicateur	Référence simulée	Pilote simulé	Lecture du résultat
Temps humain moyen	12 min	9 min	Baisse de 25 %, cible atteinte
Réouvertures à sept jours	40 / 1 000, soit 4 %	70 / 1 000, soit 7 %	Hausse de 3 points, critère manqué
Coût complet par demande	2,40 €	2,10 €	Sous le plafond fixé

Le calcul suivant vérifie les deux écarts qui commandent la décision.

reference_minutes, pilot_minutes = 12, 9
reference_reopened, pilot_reopened = 40, 70
requests_per_group = 1_000

reduction = (reference_minutes - pilot_minutes) / reference_minutes
delta = (pilot_reopened - reference_reopened) / requests_per_group

print(f"Réduction du temps : {reduction:.0%}")
print(f"Écart de réouverture : {delta * 100:+.1f} points")
print("Critères descriptifs temps et qualité :",
      "atteints" if reduction >= 0.20 and delta <= 0 else "non atteints")

Sortie obtenue en exécutant ce calcul sur les données simulées :

Réduction du temps : 25%
Écart de réouverture : +3.0 points
Critères descriptifs temps et qualité : non atteints

Le gain de temps ne suffit pas à autoriser l’extension. Les critères ont été fixés ensemble : atteindre la cible de temps ne compense pas une hausse des réouvertures. Le calcul décrit les écarts ; il n’établit pas leur cause et ne quantifie pas leur incertitude. La composition des demandes ou l’expérience des agents peuvent avoir changé.

Quand c’est possible, une répartition aléatoire entre deux fonctionnements comparables aide à estimer l’effet du changement. Sinon, il faut documenter les autres explications et limiter la portée des conclusions. Le guide du Trésor britannique sur la qualité de l’évaluation d’impact développe cette distinction entre suivi et attribution. Un petit échantillon sans dégradation visible ne prouve pas non plus que la qualité est maintenue. Le protocole doit prévoir une durée et un volume adaptés à la précision recherchée.

Les résultats doivent aussi être examinés par catégorie de demande et par expérience des agents. L’étude Generative AI at Work, publiée en 2025, analyse le déploiement progressif d’une assistance auprès de 5 172 agents. Elle estime une hausse moyenne de 15 % des demandes résolues par heure. Les moins expérimentés améliorent vitesse et qualité. Les plus expérimentés gagnent peu en vitesse et connaissent une légère baisse de qualité. Ce résultat concerne un contexte précis. Il ne fixe aucune cible universelle et n’évalue pas l’efficacité du PDCA.

Enfin, la fréquence de suivi dépend de l’indicateur. Une exposition de données demande une réaction rapide. Une réouverture à sept jours exige du recul. Le NIST prévoit des revues et un suivi après déploiement avec des fréquences définies. Pour notre cas, nous proposons des alertes sur les incidents, une revue hebdomadaire des conversations et un bilan à l’échéance prévue.

Agir conduit à généraliser, corriger ou arrêter

Le bilan simulé ne justifie pas une généralisation. L’équipe conserve le périmètre limité et examine les demandes rouvertes. Les brouillons répondent-ils à une partie seulement du problème ? Les sources sont-elles périmées ? La clôture intervient-elle trop tôt ? Ce sont des hypothèses à vérifier sur les conversations, pas des causes déjà établies.

Si les vérifications révèlent des sources périmées, le cycle suivant peut porter sur leur mise à jour et leur sélection. Si elles révèlent des réponses incomplètes, il peut tester un contrôle avant clôture. Changer de modèle n’est qu’une intervention possible parmi d’autres.

Trois décisions restent ouvertes à chaque bilan :

Généraliser progressivement si les résultats et leur solidité le justifient, avec les contrôles et les moyens nécessaires.
Corriger puis retester si une nouvelle hypothèse mérite un essai limité.
Arrêter l’assistance si les risques, les coûts ou les résultats ne justifient pas sa poursuite.

Agir signifie aussi mettre à jour les procédures, former l’équipe et nommer la personne qui vérifiera les effets de la décision. Les heures libérées restent une capacité disponible tant que leur usage n’est pas établi. Le billet sur le retour sur investissement logiciel explique comment éviter de les compter comme des économies déjà réalisées.

Après extension, le suivi continue. Un nouveau modèle, une documentation modifiée ou une nouvelle population de clients peuvent changer les résultats. Le NIST prévoit notamment le suivi, les retours des utilisateurs et les mécanismes de retrait dans la gestion après déploiement. Pour notre équipe, chaque changement important déclenche une nouvelle évaluation avant extension. Relancer le PDCA peut donc conduire à réviser le processus, les critères ou la solution technique.

Un premier cycle commence par une fiche de décision

L’équipe du cas fictif a sa réponse : le temps baisse, mais le critère de qualité est manqué. Elle doit comprendre les réouvertures et tester une correction avant d’étendre l’assistance. Voici la fiche qu’elle prépare avant ce nouvel essai.

À écrire avant l’essai	Question à trancher
Problème et périmètre	Pour quelles demandes, quelles personnes et quelle difficulté ?
Hypothèse et référence	Quel changement devrait agir sur quel indicateur, comparé à quoi ?
Critères de décision	Quel gain attendu, quelles limites de qualité et de coût ?
Protocole et échéance	Quelle comparaison, quel volume, quelle fenêtre de suivi et quelle date de bilan ?
Responsabilités	Qui met en œuvre, contrôle, analyse et décide ?
Retour au fonctionnement précédent	Comment suspendre l’assistance et continuer le service ?
Décision et apprentissage	Que généraliser, corriger ou arrêter, et que vérifier ensuite ?

Avant votre prochain pilote d’IA, écrivez le résultat qui vous ferait renoncer à le généraliser. Cette phrase rend la phase « Vérifier » utile avant même que la phase « Faire » commence.

Références

American Society for Quality, sans date. Voir What Is the Plan-Do-Check-Act Cycle?.
The W. Edwards Deming Institute, sans date. Voir PDSA Cycle.
NIST, 2023, version 1.0 ; révision en cours lors de la consultation. Voir AI RMF Core.
HM Treasury et Evaluation Task Force, mise à jour du 15 mai 2026. Voir Quality in Policy Impact Evaluation.
Erik Brynjolfsson, Danielle Li et Lindsey Raymond, 2025. Voir Generative AI at Work, The Quarterly Journal of Economics.

Sources consultées le 3 novembre 2026.

5W2H (QQOQCCP) : sept questions qui rendent un plan d'action exécutable, et celle qui a trois sens

Mon, 02 Nov 2026 00:00:00 GMT

Mots-clés : 5W2H, QQOQCCP, plan d'action, PDCA, 5 Pourquoi, diagramme d'Ishikawa, gestion de la qualité, postmortem

Décembre 2025, fin de la revue stratégique annuelle d'une équipe d'ingénierie. En octobre, un certificat expiré a fait refuser environ 1 800 paiements. L'objectif est donc clair : « aucune panne due à un certificat expiré en 2026 ». Le plan d'action tient en une ligne : « Améliorer la surveillance des certificats, équipe plateforme, T1 ». Trois mois plus tard, la ligne n'a pas bougé. Personne ne savait qui devait commencer, par où, ni avec quel budget. Le 5W2H, ou QQOQCCP en français, est l'une des grilles les plus répandues contre ce problème. Chaque action doit répondre à sept questions : quoi, pourquoi, où, quand, qui, comment et combien. Cet article montre leurs deux usages, la question qui a trois sens, et ce que la grille ne fait pas. Vous repartirez avec un modèle de ligne et un petit script qui signale les cases vides ou floues d'un plan.

Une action sans responsable ni date n'est qu'un souhait

Le scénario de cet article est fictif, mais chacun de ses éléments est courant. Le 14 octobre 2025 à 02:10, le certificat TLS (Transport Layer Security) d'une passerelle interne, pay-gw, expire. Cette passerelle relie l'API (Application Programming Interface (interface de programmation)) publique au service de paiement. Aucune alerte ne se déclenche. Pendant deux heures, environ 1 800 paiements sont refusés, jusqu'à ce que l'astreinte trouve la cause et remplace le certificat à 04:12. Le postmortem constate que personne ne savait que ce certificat existait. En décembre, la revue stratégique en fait un objectif pour 2026.

La ligne du plan d'action nomme une intention, une équipe et un trimestre. Chacun de ces trois éléments laisse une question ouverte :

« Améliorer » ne dit pas à quel moment le travail est terminé.
« Équipe plateforme » désigne six personnes. Aucune n'est chargée de commencer.
« T1 » fixe une période, pas une échéance. Le retard n'apparaît que le 31 mars, quand il est trop tard pour réagir.
Rien ne dit pourquoi cette action, où, comment, ni avec quel budget.

Le livre Site Reliability Engineering de Google définit le postmortem comme le compte rendu écrit d'un incident, qui se termine par les actions de suivi destinées à éviter qu'il se reproduise. Ce sont ces actions qui se perdent quand elles sont écrites comme la ligne de décembre. Qu'est-ce qu'une ligne doit contenir pour qu'on puisse l'exécuter et la relancer ? Le 5W2H répond par sept questions. Pourquoi ces sept-là ? La réponse commence bien avant la gestion de la qualité.

Les questions ont 2 300 ans ; le « combien » n'a pas d'origine connue

Au IVe siècle avant notre ère, Aristote se demande dans l'Éthique à Nicomaque (livre III, chapitre 1) ce qu'une personne peut ignorer de son propre acte. Il en dresse la liste : qui agit, ce qu'elle fait, sur quoi ou sur qui, avec quel instrument, en vue de quoi et comment. Vers l'an 95, Quintilien classe dans l'Institution oratoire (V, 10) les arguments « tirés des circonstances » : pourquoi, où, quand, comment et par quels moyens un acte a été accompli.

Au XIIIe siècle, Thomas d'Aquin cite dans la Somme théologique (I-II, q. 7, a. 3) un vers latin qui réunit sept circonstances : Quis, quid, ubi, quibus auxiliis, cur, quomodo, quando, soit qui, quoi, où, par quels moyens, pourquoi, comment et quand. Il l'attribue à Cicéron, tandis que Michael Sloan, dans Classical Philology en 2010, fait remonter la liste à Aristote. Personne ne peut en désigner un inventeur unique.

En avril 1900, Rudyard Kipling publie « The Elephant's Child » dans le Ladies' Home Journal. Le conte entre en 1902 dans les Just So Stories, suivi d'un poème qui nomme six de nos sept questions :

I Keep six honest serving-men:
(They taught me all I knew)
Their names are What and Where and When
And How and Why and Who.

En traduction libre : « J'ai six honnêtes serviteurs (ils m'ont appris tout ce que je savais) ; ils s'appellent Quoi, Où, Quand, Comment, Pourquoi et Qui. » La seconde strophe parle d'une « petite personne » qui en garde dix millions : « un million de Comment, deux millions de Où, et sept millions de Pourquoi ». Déjà chez Kipling, le pourquoi est la question qui prend le plus de place.

En 1913, le manuel Newspaper Writing and Editing de Willard Bleyer demande que le premier paragraphe d'un article de presse réponde aux questions What? Who? When? Where? Why? How? Ce sont les « 5W1H » du journalisme. Six questions suffisent pour raconter un fait.

Pour planifier, il en manque une. La fiche de l'AHRQ (Agency for Healthcare Research and Quality (agence fédérale américaine pour la recherche et la qualité en santé)) sur le 5W2H ajoute How much, combien, et cite comme source The Quality Toolbox de Nancy Tague (ASQ (American Society for Quality) Quality Press, 2005). Le second H est la question des ressources : temps, argent, personnes. Un journaliste peut s'en passer. Un plan d'action, non : c'est cette question qui permet de comparer deux actions. C'est aussi la case qui manquait à la ligne de décembre : personne ne savait avec quel budget.

Quand, et par qui, ce « combien » a-t-il été ajouté ? Je n'ai trouvé aucune source fiable pour le dire. On lit souvent que la grille viendrait de Toyota ou de la qualité totale japonaise, mais ces attributions ne renvoient à aucun texte primaire.

En français, la grille s'appelle QQOQCCP : qui, quoi, où, quand, comment, combien, pourquoi. La fiche de Techniques de l'Ingénieur la présente comme une série de questions standard pour clarifier une situation mal connue. Le second C est parfois omis, ce qui donne QQOQCP. L'ordre des lettres ne compte pas : la fiche de l'AHRQ précise que l'ordre des questions n'a pas d'importance.

Le même tableau sert à décrire un problème ou à planifier une action

La fiche de l'AHRQ énumère plusieurs usages : planifier un projet, améliorer un processus, définir un problème soupçonné ou mal cerné. Dans tous les cas, on répond à chaque question, et on prévoit comment répondre à celles qui n'ont pas encore de réponse. Deux usages dominent, et ils ne regardent pas dans la même direction.

Décrire un problème consiste à poser les questions sur un fait passé. C'est l'étape D2 de la méthode 8D. Christoph Roser, professeur et auteur sur le lean, précise dans son guide du 8D le sens de trois questions. Le pourquoi y désigne l'impact, et pas encore la cause racine. Le comment décrit l'écart observé par rapport à l'état normal. Le combien quantifie le problème : taux de défauts, réclamations, coût des rebuts.

Planifier une action consiste à poser les questions sur un fait futur, une ligne par action. Au début de la pandémie de COVID-19, une équipe infirmière d'un hôpital universitaire de Rio de Janeiro a d'abord classé les problèmes avec un diagramme d'Ishikawa. Elle a ensuite construit son plan avec le 5W2H : sept actions, arrêtées en six réunions du 16 mars au 31 mai 2020. Le cas est décrit par Braga et al. (2023).

La figure applique les deux usages au scénario. À gauche, la grille décrit la panne du 14 octobre. À droite, elle planifie l'action A2, l'alerte avant expiration.

Figure: Les mêmes sept questions décrivent la panne et planifient l'action A2. Seul le pourquoi change de nature : l'impact à gauche, la justification à droite. (interactive version in the post)

Les six autres lignes changent d'objet, pas de nature. Le quand reste une date : passée à gauche, échéance à droite. Le combien reste une quantité : des paiements refusés à gauche, des jours-personne à droite. Le pourquoi, lui, ne répond plus à la même question. Mélanger les deux colonnes produit des plans qui décrivent au lieu d'engager. On y lit par exemple « Pourquoi : 2 h sans paiement » en face d'une action, sans savoir quelle cause elle traite.

Le « pourquoi » a trois sens selon le moment où on le pose

Quand on décrit, le pourquoi demande en quoi la situation est un problème : la réponse est l'impact. Quand on planifie, il demande pourquoi cette action plutôt qu'une autre : la réponse est une justification. Entre les deux, une troisième méthode pose la même question plusieurs fois pour trouver la cause : les 5 Pourquoi.

Le Lean Enterprise Institute rappelle l'exemple classique de Taiichi Ohno, l'un des architectes du système de production de Toyota (Ballé, 2012). Une machine s'arrête. Pourquoi ? Une surcharge a fait sauter le fusible. Pourquoi la surcharge ? Le palier était mal lubrifié. Pourquoi ? La pompe de lubrification pompait mal. Pourquoi ? Son axe était usé. Pourquoi ? Il n'y avait pas de filtre, et des copeaux de métal sont entrés. Remplacer le fusible traiterait le symptôme. Poser un filtre traite la cause.

Appliquée à la panne du 14 octobre, la chaîne donne ceci :

Pourquoi les paiements ont-ils échoué ? Le certificat de pay-gw avait expiré.
Pourquoi a-t-il expiré ? Personne ne l'a renouvelé.
Pourquoi ? Il ne figurait dans aucun inventaire, et son renouvellement était manuel.
Pourquoi la panne a-t-elle duré deux heures ? Rien ne surveillait la date d'expiration, et l'astreinte a dû chercher la cause.

La chaîne aboutit à trois causes : un certificat hors inventaire, un renouvellement manuel et une expiration que rien ne surveillait. Voici la réponse à la question ouverte au début : dans un plan d'action, le pourquoi de chaque ligne doit citer la cause qu'elle traite. « 2 h sans paiement » justifie l'urgence de l'objectif, pas le choix d'une action. Un pourquoi qui ne cite aucune cause signale une action qui traite un symptôme, ou rien du tout.

Les 5 Pourquoi ont leurs propres limites. Michael Ballé note que beaucoup de séances restent superficielles ou tournent en rond, et que leur valeur dépend de l'expertise de ceux qui les mènent. Alan Card, dans BMJ Quality & Safety en 2017, avance que la méthode doit « probablement » sa place dans l'analyse des causes à son pedigree, à sa simplicité et à sa valeur pédagogique.

Dans le PDCA, le 5W2H vient après l'analyse des causes, pas avant

Les trois sens du pourquoi correspondent à trois moments d'un même cycle. Le PDCA (Plan, Do, Check, Act (planifier, faire, vérifier, agir)) enchaîne quatre phases : planifier un changement, le mettre en œuvre, évaluer les résultats, puis standardiser ou recommencer. Le Lean Enterprise Institute attribue le premier cycle d'amélioration à Walter Shewhart, et son introduction au Japon, dans les années 1950, à W. Edwards Deming. La phase Plan contient à elle seule les trois usages du pourquoi.

Figure: Dans la phase Plan, on décrit, puis on explique, puis on planifie. Le 5W2H arrive en troisième, quand les causes sont connues. (interactive version in the post)

L'hôpital de Rio suit cet ordre : l'Ishikawa d'abord, le 5W2H ensuite. Dans une étude de cas industrielle, Mariani (2005) utilise aussi le 5W2H à l'intérieur d'un PDCA, avec le diagramme de cause à effet. Planifier avant d'expliquer revient à choisir des actions avant de connaître les causes. La ligne de décembre est née ainsi, d'un objectif et non d'une analyse.

Il manque pourtant à la grille une case que la norme exige. La clause 6.2.2 de l'ISO 9001:2015 demande, pour chaque objectif qualité, de déterminer ce qui sera fait, les ressources nécessaires, le responsable, l'échéance et la manière d'évaluer les résultats. Les quatre premiers points correspondent au quoi, au combien, au qui et au quand. Le cinquième n'a pas d'équivalent dans le 5W2H. Pour A2, ce critère pourrait être : chaque certificat de l'inventaire A1 a une alerte, et un certificat de test qui expire dans 29 jours déclenche l'alerte en préproduction.

Le tableau d'actions de l'exemple de postmortem du livre SRE (Site Reliability Engineering) a, lui, quatre colonnes : Action Item, Type, Owner et Bug. Il couvre le quoi, le qui et le suivi par un ticket. Le type, « atténuer » ou « prévenir », est un pourquoi très court. Le quand et le combien manquent. Si votre modèle de postmortem reprend ce tableau, ajoutez deux colonnes : l'échéance et l'effort estimé.

Réécrire le plan ligne par ligne fait apparaître ce qui manque

Le 16 mars 2026, l'équipe réécrit la ligne de décembre. Les trois causes deviennent trois actions, et chaque action répond aux sept questions :

Question	A1	A2	A3
quoi	Inventorier les certificats TLS de production	Alerter l'astreinte 30 jours avant chaque expiration	Automatiser le renouvellement des certificats
pourquoi	Le certificat expiré n'était dans aucun inventaire	L'expiration n'a déclenché aucune alerte	Le renouvellement était manuel
où	Répartiteurs de charge et passerelles internes	Supervision Prometheus existante	Cluster Kubernetes de production
quand	3 avril 2026	24 avril 2026	30 juin 2026
qui	Camille	Sacha	Dominique
comment	Script sur l'API du cloud, export CSV versionné	Sonde TLS du blackbox exporter, règle d'alerte	cert-manager avec ACME, d'abord en préproduction
combien	3 jours-personne	2 jours-personne	à définir

A2 repose sur la métrique probe_ssl_earliest_cert_expiry du blackbox exporter, qui donne la date d'expiration du premier certificat de la chaîne à expirer. A3 repose sur cert-manager, qui renouvelle les certificats dans le cluster Kubernetes avec le protocole ACME (Automatic Certificate Management Environment), décrit dans la RFC 8555.

Une relecture peut vérifier ce tableau. Un script aussi, au moins pour la forme. Le programme ci-dessous applique cinq règles à chaque ligne :

aucune des sept cases n'est vide ;
le quoi ne commence pas par un verbe flou, comme « améliorer » ou « renforcer » ;
le qui nomme une personne, pas une équipe ni une liste ;
le quand est une date, postérieure au jour de la réécriture ;
le combien contient au moins un chiffre.

Le bouton Run l'exécute dans votre navigateur. Vous pouvez modifier le plan et le relancer.

import re
from datetime import date

# Jour de la réécriture : chaque échéance doit tomber après.
AUJOURD_HUI = date(2026, 3, 16)

QUESTIONS = ("quoi", "pourquoi", "où", "quand", "qui", "comment", "combien")
VERBES_FLOUS = ("améliorer", "renforcer", "optimiser", "suivre", "garantir")
COLLECTIFS = ("équipe", "tous", "direction", " et ", ",", "/")


def verifier(action):
    """Renvoie la liste des problèmes d'une ligne du plan."""
    problemes = []
    for question in QUESTIONS:
        if not action.get(question, "").strip():
            problemes.append(f"{question} : vide")

    quoi = action.get("quoi", "").lower()
    if quoi.startswith(VERBES_FLOUS):
        problemes.append(f"quoi : verbe flou « {quoi.split()[0]} »")

    qui = action.get("qui", "")
    if qui and any(mot in f" {qui.lower()} " for mot in COLLECTIFS):
        problemes.append(f"qui : « {qui} » n'est pas une personne")

    quand = action.get("quand", "")
    if quand:
        try:
            if date.fromisoformat(quand) <= AUJOURD_HUI:
                problemes.append(f"quand : {quand} est déjà passé")
        except ValueError:
            problemes.append(f"quand : « {quand} » n'est pas une date")

    combien = action.get("combien", "")
    if combien and not re.search(r"\d", combien):
        problemes.append(f"combien : « {combien} » sans chiffre")
    return problemes


PLAN = {
    "A0": {
        "quoi": "Améliorer la surveillance des certificats",
        "qui": "équipe plateforme",
        "quand": "T1",
    },
    "A1": {
        "quoi": "Inventorier les certificats TLS de production",
        "pourquoi": "Le certificat expiré n'était dans aucun inventaire",
        "où": "Répartiteurs de charge et passerelles internes",
        "quand": "2026-04-03",
        "qui": "Camille",
        "comment": "Script sur l'API du cloud, export CSV versionné",
        "combien": "3 jours-personne",
    },
    "A2": {
        "quoi": "Alerter l'astreinte 30 jours avant chaque expiration",
        "pourquoi": "L'expiration n'a déclenché aucune alerte",
        "où": "Supervision Prometheus existante",
        "quand": "2026-04-24",
        "qui": "Sacha",
        "comment": "Sonde TLS du blackbox exporter et règle d'alerte",
        "combien": "2 jours-personne",
    },
    "A3": {
        "quoi": "Automatiser le renouvellement des certificats",
        "pourquoi": "Le renouvellement était manuel",
        "où": "Cluster Kubernetes de production",
        "quand": "2026-06-30",
        "qui": "Dominique",
        "comment": "cert-manager avec ACME, d'abord en préproduction",
        "combien": "à définir",
    },
}

for nom, action in PLAN.items():
    problemes = verifier(action)
    print(f"{nom} : {len(problemes)} problème(s)")
    for probleme in problemes:
        print(f"  - {probleme}")

Voici sa sortie, avec Python 3.14 :

A0 : 7 problème(s)
  - pourquoi : vide
  - où : vide
  - comment : vide
  - combien : vide
  - quoi : verbe flou « améliorer »
  - qui : « équipe plateforme » n'est pas une personne
  - quand : « T1 » n'est pas une date
A1 : 0 problème(s)
A2 : 0 problème(s)
A3 : 1 problème(s)
  - combien : « à définir » sans chiffre

La ligne de décembre, A0, échoue sur les sept questions : quatre cases vides et trois réponses qui n'en sont pas. A1 et A2 passent. A3 échoue sur une seule case, la plus facile à remplir pour la forme. Personne ne sait encore combien coûte l'automatisation des certificats internes, et c'est une information en soi. La correction n'invente pas de chiffre : elle planifie l'estimation. La case devient « inconnu ; estimation par Dominique le 15 avril 2026 ». C'est ce que recommande la fiche de l'AHRQ : prévoir comment répondre aux questions dont on ne connaît pas encore la réponse.

Cette nouvelle case passe la règle du chiffre grâce à la date, pas grâce à un coût. Le script vérifie la forme, pas le fond. Une ligne peut passer les cinq règles et rester une mauvaise action. C'est aussi la limite de la grille elle-même.

La grille ne priorise pas, et rien ne prouve son effet

La fiche de l'AHRQ ne lui reconnaît qu'un inconvénient : elle n'est pas utile si la plupart des réponses sont inconnues. Elle affirme aussi que la grille prend en compte tous les aspects d'une situation. Elle en couvre sept. La priorité, le risque, les dépendances entre actions et la force d'une action n'en font pas partie.

La grille ne dit pas quelle action est la plus forte. A2 et A3 passent le script avec zéro problème, mais elles ne se valent pas. Le guide RCA² (Root Cause Analysis and Action (analyse des causes racines et actions)) de la National Patient Safety Foundation (2015) classe les actions correctives selon une hiérarchie. Les plus fortes retirent l'occasion de l'erreur : contraintes techniques, simplification, standardisation. Les plus faibles dépendent de quelqu'un qui se souvient et obéit : avertissements, nouvelles procédures, formation. L'outil de hiérarchie des actions demande au moins une action forte ou intermédiaire par cause. L'alerte A2 est un avertissement : elle compte sur une astreinte qui lit et agit. A3 supprime l'étape humaine. A3 est donc plus forte, et la grille ne le montre pas.

Les cases se remplissent pour la forme. Dans le plan de l'hôpital de Rio, la colonne How much indique « Unestimated Expense » pour les sept actions. Aucune n'a de coût. L'article ne mesure pas non plus de résultat, et ses auteurs reconnaissent l'absence de cadre méthodologique structuré, puisqu'il s'agit d'un retour d'expérience.

Aucune étude contrôlée ne mesure son effet. Je n'ai trouvé aucune étude qui compare des équipes avec et sans 5W2H. La littérature disponible se compose de retours d'expérience et d'études de cas uniques, comme celles de Braga et de Mariani. La grille est sans doute utile, mais personne ne l'a démontré.

La grille ne choisit pas l'objectif. « Aucune panne due à un certificat expiré » vient du postmortem et de la revue stratégique. Le 5W2H décline cet objectif en actions. Il ne dit pas s'il valait mieux investir ailleurs. Une grille parfaite exécute aussi bien un mauvais objectif qu'un bon. Pour mettre à l'épreuve les hypothèses d'un objectif, le registre de prémisses de l'article sur le método científico (en espagnol) propose une grille voisine : une ligne par affirmation, et le résultat qui la réfuterait.

Les réponses coûtent plus cher que les cases. La fiche de Techniques de l'Ingénieur avertit de ne pas sous-estimer le temps de recueil des informations. Écrire sept cases prend quelques minutes. Trouver le coût réel de A3 demande une estimation, et donc une action de plus dans le plan.

Une action exécutable nomme une personne, une date et un coût

Une action est exécutable quand la personne nommée peut commencer le lendemain sans poser de question. La ligne de décembre ne répondait de façon exploitable à aucune des sept questions. Réécrite, elle devient trois lignes. Chacune nomme une personne, une date et un effort, ou la date à laquelle on le connaîtra. Son pourquoi cite une cause trouvée par l'analyse. Rien ne garantit que les trois actions réussiront. Mais chacune a désormais quelqu'un à relancer, une date où l'on saura, et un coût que l'on peut comparer aux autres.

La question qui a trois sens est le pourquoi : l'impact quand on décrit, la cause quand on explique, la justification quand on planifie. Dans un plan, il doit renvoyer à une cause.

À retenir, et à appliquer dès la prochaine réunion : avant de la clore, relisez chaque action et demandez qui, quand, combien, et quelle cause elle traite. Si une réponse manque, la première action à écrire est celle qui la trouvera, avec un nom et une date.

Références

Origines des questions : Aristote, Éthique à Nicomaque, livre III, trad. W. D. Ross ; Quintilien, Institution oratoire, livre V, trad. H. E. Butler (Loeb, 1920-1922) ; Thomas d'Aquin, Somme théologique, I-II, q. 7 ; M. C. Sloan, « Aristotle's Nicomachean Ethics as the Original Locus for the Septem Circumstantiae », Classical Philology, 2010.
Kipling et le journalisme : R. Kipling, Just So Stories (1902) ; Kipling Society, notes sur The Elephant's Child ; W. G. Bleyer, Newspaper Writing and Editing (1913), chapitre IV.
La méthode : AHRQ, 5W2H, d'après N. Tague, The Quality Toolbox, 2e éd., ASQ Quality Press, 2005 ; I. Equixor, La méthode QQOQCCP, un outil d'analyse, Techniques de l'Ingénieur, fiche FIC0447 ; C. Roser, The Eight Disciplines Problem Solving (8D): D2 and D3, AllAboutLean (source secondaire).
Études de cas : Braga, Lins, Christovam et de Souza, « Quality management in the COVID-19 pandemic: nursing action plan », Revista Brasileira de Enfermagem, 2023 ; C. A. Mariani, « Método PDCA e ferramentas da qualidade no gerenciamento de processos industriais », RAI, 2005.
Causes, cycle et force des actions : M. Ballé, Five whys, Lean Enterprise Institute, 2012 ; A. J. Card, « The problem with '5 whys' », BMJ Quality & Safety, 2017 ; Lean Enterprise Institute, PDCA ; norme ISO 9001:2015, clause 6.2.2 ; NPSF et IHI, RCA²: Improving Root Cause Analyses and Actions to Prevent Harm (2015), et son outil de hiérarchie des actions.
Logiciel : J. Lunney et S. Lueder, Postmortem Culture: Learning from Failure et Example Postmortem, Site Reliability Engineering, Google, 2016 ; Prometheus, blackbox exporter ; documentation de cert-manager ; R. Barnes et al., RFC 8555 (2019).

ASGI Event by Event: How scope, receive and send Keep a Connection Open, and Why an Async WSGI Would Not

Mon, 02 Nov 2026 00:00:00 GMT

Keywords: ASGI, WSGI, scope, receive, send, Uvicorn, WebSocket, long polling, lifespan, asgiref

An order page should show "shipped" the moment the warehouse marks order 42 as shipped. To do that, the server has to send a message long after the browser asked, on a connection that stays open. A WSGI (Web Server Gateway Interface) application cannot do this, because it is a function that takes one request and returns one response. Making that function async would not fix it either, and the ASGI introduction explains why. This post builds the feature on raw ASGI (Asynchronous Server Gateway Interface), with no framework, runs it under Uvicorn, and prints every event that passes between the server and the application. You will see what scope, receive and send carry, how a request body arrives in pieces, and how the application learns that a client has left. You will also see what happens to old WSGI code on the same server, and why three one-second WSGI requests took three seconds.

The WSGI vs ASGI post showed a hello-world application for each interface, and when to choose each one. This post goes one level down, to the messages themselves. Every trace below is real output from Python 3.13.9, Uvicorn 0.54.0 with its h11 HTTP implementation, asgiref 3.13.0, httpx 0.28.1 and websockets 17.4. It ran on one machine, over localhost, with one worker process. A real deployment adds a network, several workers and a message broker, and the sections say where that changes the result.

WSGI hands the application one request and takes back one response

The WSGI specification, PEP (Python Enhancement Proposal) 3333, defines the application as "simply a callable object that accepts two arguments": environ, a dict that describes the request, and start_response, a function that sets the status and headers. The server "invokes the application callable once for each request", and the application returns "an iterable yielding zero or more bytestrings". Every input arrives before the call starts, and every output leaves through the return value.

The order page needs two things this shape cannot give. The first is a long poll over HTTP (Hypertext Transfer Protocol): the browser asks for the next status change, and the server answers only when there is one. In WSGI, the call must stay running for the whole wait, and nothing in the interface tells it that the browser has given up. The second is a WebSocket. After the handshake, the browser sends frames whenever it wants. A call that has already received its environ has no way to get them.

Would an async WSGI help? Picture one:

# A hypothetical async WSGI: not a real interface.
async def application(environ):
    ...  # everything the client will send is already in environ
    return [b"..."]  # everything the server will send leaves here

The async lets the server run other requests on its event loop while this one waits, but the application still has one way in and one way out. The ASGI introduction puts it this way: "Even if we made this callable asynchronous, it still only has a single path to provide a request, so protocols that have multiple incoming events (like receiving WebSocket frames) can't trigger this." The ASGI specification gives the same reason in its rationale: WSGI's design "is irrevocably tied to the HTTP-style request/response cycle".

Figure: A WSGI call gets the whole request when it starts and gives back the whole response when it returns. An ASGI call lasts as long as the connection, and events cross in both directions while it runs, including a message the app sends when nothing asked for it. (interactive version in the post)

The figure raises three questions that the rest of the post answers with a running server. How does an application read a request that is still arriving? How does it learn that the browser has gone? And can the old WSGI code of the same project live on the new server, at the same speed?

An ASGI application is called once per connection, with three arguments

ASGI started in Django Channels. Andrew Godwin proposed it on the Web-SIG list in March 2016. He argued for "a clean break from WSGI to a new structure (and NOT calling it 'WSGI 2')". Version 3.0, the current one, dates from March 2019. It retired an older style in which the application was split into two callables. Today the whole interface is one line of the specification:

coroutine application(scope, receive, send)

The specification defines the three arguments:

scope: "a dictionary that contains at least a type key specifying the protocol that is incoming". It describes the connection.
receive: "an awaitable callable that will yield a new event dictionary when one is available".
send: "an awaitable callable taking a single event dictionary as a positional argument that will return once the send has been completed or the connection has been closed".

"The application is called once per 'connection'," the specification goes on. "With HTTP it is one request, whereas for a WebSocket it is a single WebSocket connection." To see what a server actually passes, this application prints its arguments and answers 204 No Content. The output below leaves out a readiness probe and shortens the module paths:

# scope_dump.py
from pprint import pprint


async def app(scope, receive, send):
    if scope["type"] != "http":
        raise RuntimeError(f"unsupported scope type: {scope['type']}")
    pprint(scope)
    print(receive, send, sep="\n")
    await send({"type": "http.response.start", "status": 204, "headers": []})
    await send({"type": "http.response.body"})

uvicorn scope_dump:app --port 8044
curl -H "accept: application/json" "http://127.0.0.1:8044/orders/42?fields=status"

INFO:     Waiting for application startup.
INFO:     ASGI 'lifespan' protocol appears unsupported.
INFO:     Application startup complete.
{'asgi': {'spec_version': '2.3', 'version': '3.0'},
 'client': ('127.0.0.1', 55899),
 'headers': [(b'host', b'127.0.0.1:8044'),
             (b'user-agent', b'curl/8.7.1'),
             (b'accept', b'application/json')],
 'http_version': '1.1',
 'method': 'GET',
 'path': '/orders/42',
 'query_string': b'fields=status',
 'raw_path': b'/orders/42',
 'root_path': '',
 'scheme': 'http',
 'server': ('127.0.0.1', 8044),
 'state': {},
 'type': 'http'}
<bound method RequestResponseCycle.receive of <...h11_impl.RequestResponseCycle object at 0x10a5356d0>>
<bound method RequestResponseCycle.send of <...h11_impl.RequestResponseCycle object at 0x10a5356d0>>

Four details in that output come back later:

The body is missing. The scope holds the method, the path, the query and the headers, but the specification leaves out "the HTTP request body, as this is streamed in via events".
The types are exact. The path is a str, decoded. The query string and the headers are bytes, as the client sent them. Header names are lowercase.
Two versions are given. version is the ASGI version, 3.0. spec_version is the version of the HTTP message format that this code path of Uvicorn implements, 2.3. The HTTP and WebSocket format is at 2.5. The difference matters below, in the section on disconnects.
receive and send belong to one request. They are two methods of the same RequestResponseCycle object, which Uvicorn creates for each request.

The first log lines are a result of the raise. Before the first request, Uvicorn called the application with a lifespan scope. The application raised, as the specification asks: "Applications should actively reject any protocol that they do not understand with an Exception". Uvicorn then went on without lifespan events. The lifespan section below accepts that scope.

Every event is a dict, and its `type` key names its shape

Each event is a plain dict. Its type is "namespaced as protocol.message_type", so http.request belongs to the HTTP protocol and websocket.send to the WebSocket protocol. These are the core events of the three protocols:

Protocol	From `receive()`	To `send()`
HTTP	`http.request`, `http.disconnect`	`http.response.start`, `http.response.body`
WebSocket	`websocket.connect`, `websocket.receive`, `websocket.disconnect`	`websocket.accept`, `websocket.send`, `websocket.close`
Lifespan	`lifespan.startup`, `lifespan.shutdown`	`lifespan.startup.complete` or `.failed`, `lifespan.shutdown.complete` or `.failed`

Because every event passes through two callables, a wrapper around them sees all the traffic. The specification calls such a wrapper middleware: "code that plays the role of both server and application". This one replaces receive and send with versions that print each event before passing it on. <- marks an event that receive() returned, -> an event given to send(), and ** a new call of the application:

# tracing.py
import threading
import time

_start = _last = time.monotonic()


def clock():
    """Milliseconds since the experiment began, after a second of silence."""
    global _start, _last
    now = time.monotonic()
    if now - _last > 1:
        _start = now
    _last = now
    return (now - _start) * 1000


def log(arrow, text):
    print(f"{clock():6.1f} ms {arrow} {text}", flush=True)


def show(event):
    fields = " ".join(f"{k}={v!r}" for k, v in event.items() if k != "type")
    return f"{event['type']} {fields}".rstrip()


def trace(app):
    async def traced(scope, receive, send):
        async def traced_receive():
            event = await receive()
            log("<-", show(event))
            return event

        async def traced_send(event):
            log("->", show(event))
            await send(event)

        parts = [scope["type"], "scope", scope.get("method"), scope.get("path")]
        if scope.get("client"):
            parts.append(f"port {scope['client'][1]}")
        parts.append(f"in {threading.current_thread().name}")
        log("**", " ".join(p for p in parts if p))
        await app(scope, traced_receive, traced_send)

    return traced

The orders application routes on scope["type"] first and on the path second. It is about a hundred lines, and the sections below show its parts:

# orders.py (routing)
async def app(scope, receive, send):
    if scope["type"] == "lifespan":
        await lifespan(scope, receive, send)
    elif scope["type"] == "websocket":
        await order_feed(scope, receive, send)
    elif scope["type"] != "http":
        raise RuntimeError(f"unsupported scope type: {scope['type']}")
    elif scope["path"] == "/orders/42/status":
        await update_status(scope, receive, send)
    elif scope["path"] == "/orders/42/next":
        await wait_for_update(scope, receive, send)
    elif scope["path"].startswith("/legacy"):
        await legacy(scope, receive, send)
    else:
        await respond(send, 404, {"error": "not found"})


traced = trace(app)

uvicorn orders:traced --port 8042 --no-access-log

A body can arrive in several events, so read until `more_body` is false

The warehouse reports a status change with a POST to /orders/42/status. To show how a body arrives, the test client sends this one in three pieces, 100 ms apart. httpx sends a body that it reads from a generator with Transfer-Encoding: chunked:

async def body():
    for piece in (b'{"order": 42, ', b'"status": ', b'"packed"}'):
        yield piece
        await asyncio.sleep(0.1)

await client.post("http://127.0.0.1:8042/orders/42/status", content=body())

   0.0 ms ** http scope POST /orders/42/status port 55985 in MainThread
   0.1 ms <- http.request body=b'{"order": 42, ' more_body=True
 106.7 ms <- http.request body=b'"status": ' more_body=True
 211.5 ms <- http.request body=b'"packed"}' more_body=True
 313.6 ms <- http.request body=b'' more_body=False
 314.0 ms -> http.response.start status=202 headers=[(b'content-type', b'application/json')]
 314.4 ms -> http.response.body body=b'{"listeners": 0}'

Uvicorn removed the chunk framing and passed each piece on as an http.request event. The specification gives the rule for more_body: "If True, the consuming application should wait until it gets a chunk with this set to False." The last event is empty. It arrived when the client's generator finished and the client sent the final zero-length chunk, about 100 ms after the last piece. Only then does the server know that the body is complete. The handler reads every event and joins the pieces:

async def read_body(receive):
    body = b""
    while True:
        event = await receive()
        if event["type"] == "http.disconnect":
            raise ConnectionError("client left before the body ended")
        body += event.get("body", b"")
        if not event.get("more_body", False):
            return body


async def respond(send, status, payload):
    await send({
        "type": "http.response.start",
        "status": status,
        "headers": [(b"content-type", b"application/json")],
    })
    await send({"type": "http.response.body", "body": json.dumps(payload).encode()})


async def update_status(scope, receive, send):
    update = json.loads(await read_body(receive))
    publish(update)
    await respond(send, 202, {"listeners": len(subscribers)})

The response has the same two-part shape. http.response.start carries the status and the headers. The specification says it "needs to be followed by at least one response content message", and the server may hold it back until the first http.response.body. An http.response.body without more_body ends the response. {"listeners": 0} means that nobody was waiting for order 42 yet. The next section adds a listener.

The client port also says something. The next two requests in the lab reused port 55985, so httpx kept the same TCP (Transmission Control Protocol) connection open for them. Each request still got its own call and its own scope. The HTTP format states that an HTTP connection scope lasts one request, "even if the underlying socket is still open and serving multiple requests".

`receive()` is also how the application learns the client has left

The long poll is GET /orders/42/next. The handler subscribes to updates and waits for whichever comes first: the next update, or the browser leaving. The event for the second case is http.disconnect, which the format describes as "mainly useful for long-polling, where you may want to trigger cleanup code if the connection closes early". The application gets it from receive(), so the handler waits on two tasks at once with asyncio.wait.

My first version did exactly that and failed. The client got 500 Internal Server Error, and the trace showed why:

   0.0 ms ** http scope GET /orders/42/next in MainThread
   0.1 ms <- http.request body=b'' more_body=False
ERROR:    ASGI callable returned without starting response.

The first receive() of an HTTP call returns the request body, even for a GET with no body: an empty http.request. That event finished the "disconnect" task at once. The handler took it as the browser leaving, returned without a response, and Uvicorn answered 500 for it. The format says when http.disconnect comes: when "receive is called after a response has been sent or after the HTTP connection has been closed". Until then, receive() hands over the body. A handler that waits for a disconnect must first read the body to the end. The fix is one line, read_body before the wait:

async def wait_for_update(scope, receive, send):
    await read_body(receive)  # the first event is the request body
    queue = asyncio.Queue()
    subscribers.add(queue)
    update = asyncio.create_task(queue.get())
    disconnect = asyncio.create_task(receive())
    try:
        done, _ = await asyncio.wait(
            {update, disconnect}, return_when=asyncio.FIRST_COMPLETED
        )
        if update in done:
            await respond(send, 200, update.result())
    finally:
        subscribers.discard(queue)
        update.cancel()
        disconnect.cancel()

subscribers is a set of asyncio.Queue objects, and publish puts each update on every queue in it. With the fix, two runs show both endings. In the first, the warehouse posts "shipped" 300 ms after the browser started waiting. In the second, the browser gives up after 0.5 s:

   0.0 ms ** http scope GET /orders/42/next port 55985 in MainThread
   0.2 ms <- http.request body=b'' more_body=False
 303.1 ms ** http scope POST /orders/42/status port 55986 in MainThread
 303.2 ms <- http.request body=b'{"order":42,"status":"shipped"}' more_body=False
 303.3 ms -> http.response.start status=202 headers=[(b'content-type', b'application/json')]
 303.4 ms -> http.response.body body=b'{"listeners": 1}'
 303.5 ms -> http.response.start status=200 headers=[(b'content-type', b'application/json')]
 303.7 ms -> http.response.body body=b'{"order": 42, "status": "shipped"}'

   0.0 ms ** http scope GET /orders/42/next port 55985 in MainThread
  16.4 ms <- http.request body=b'' more_body=False
 502.1 ms <- http.disconnect

In the first run, the POST's call and the waiting GET's call ran at the same time, on one thread, and the update crossed from one to the other in less than a millisecond. In the second, http.disconnect came 502 ms after the request, when the client's 0.5 s timeout closed the socket. The handler cleaned up without sending anything.

Sending after a disconnect is the other half of the question. Since version 2.4 of the format, "If send() is called on a closed connection the server should raise a server-specific subclass of OSError". This Uvicorn path reports 2.3, and its h11 code returns from send without doing anything once the client is gone. Its WebSocket code reports 2.4 and raises ClientDisconnected, a subclass of OSError. An application that must work on any server should handle both behaviors.

A WebSocket needs a second coroutine to listen and push at once

The long poll answers once and then the browser must ask again. A WebSocket keeps one connection for the whole session. Its scope "lives as long as the socket itself - if the application dies the socket should be closed, and vice-versa". The first event is websocket.connect, and the application must answer it with websocket.accept or websocket.close before any websocket.receive can arrive. A close at this point makes the server reject the handshake with HTTP 403.

After that, the connection has two sources of events. The browser can send a frame at any time, which arrives through receive(), and the warehouse can change a status at any time, which arrives through the queue. One coroutine cannot wait on both in turn, so the handler starts a second one. The introduction describes this pattern: ASGI "allows for a background coroutine so the application can do other things (such as listening for events on an external trigger, like a Redis queue)".

async def order_feed(scope, receive, send):
    await receive()  # websocket.connect
    await send({"type": "websocket.accept"})
    queue = asyncio.Queue()
    subscribers.add(queue)

    async def push_updates():
        while True:
            update = await queue.get()
            await send({"type": "websocket.send", "text": json.dumps(update)})

    pusher = asyncio.create_task(push_updates())
    try:
        while True:
            event = await receive()
            if event["type"] == "websocket.disconnect":
                break
            await send({"type": "websocket.send", "text": f"ack {event['text']}"})
    finally:
        subscribers.discard(queue)
        pusher.cancel()

The test client opens the socket, sends hello, has the warehouse post "delivered", reads both messages and closes:

   0.0 ms ** websocket scope /orders/42/feed port 55990 in MainThread
   0.1 ms <- websocket.connect
   0.1 ms -> websocket.accept
   1.1 ms <- websocket.receive text='hello'
   1.1 ms -> websocket.send text='ack hello'
   2.0 ms ** http scope POST /orders/42/status port 55986 in MainThread
   2.0 ms <- http.request body=b'{"order":42,"status":"delivered"}' more_body=False
   2.1 ms -> http.response.start status=202 headers=[(b'content-type', b'application/json')]
   2.1 ms -> http.response.body body=b'{"listeners": 1}'
   2.2 ms -> websocket.send text='{"order": 42, "status": "delivered"}'
   2.9 ms <- websocket.disconnect code=1000 reason=''

Figure: Two calls of the same application run at once. Call A holds the browser's WebSocket, and call B serves the warehouse's POST. B puts the update on A's queue, and A's push task sends it to the browser. No request from the browser is behind that message. (interactive version in the post)

The websocket.send at 2.2 ms is the message that WSGI has no slot for. It came from the application with no request behind it. Code 1000 is "a normal closure" in RFC 6455.

The lab takes one shortcut here. The set of queues lives in the memory of one process, so it works only because Uvicorn ran one worker. With --workers 4, a POST served by one worker would never reach a WebSocket held by another. The introduction's example, a Redis queue, is what fills that gap in production: each worker subscribes to a channel outside the process.

Lifespan events give the application a start and an end of its own

Every scope so far belonged to one request or one socket. A connection pool, or the subscriber to that Redis channel, belongs to the whole process. The lifespan protocol gives the application a call for that: its messages "allow for an application to initialise and shutdown in the context of a running event loop". Uvicorn makes this call at startup, with a lifespan scope:

async def lifespan(scope, receive, send):
    while True:
        event = await receive()
        if event["type"] == "lifespan.startup":
            await send({"type": "lifespan.startup.complete"})
        elif event["type"] == "lifespan.shutdown":
            await send({"type": "lifespan.shutdown.complete"})
            return

INFO:     Waiting for application startup.
  66.1 ms ** lifespan scope in MainThread
  66.3 ms <- lifespan.startup
  66.5 ms -> lifespan.startup.complete
INFO:     Application startup complete.
INFO:     Uvicorn running on http://127.0.0.1:8042 (Press CTRL+C to quit)
...
INFO:     Shutting down
INFO:     Waiting for application shutdown.
   0.0 ms <- lifespan.shutdown
   0.1 ms -> lifespan.shutdown.complete
INFO:     Application shutdown complete.

The server "must wait for this message before it starts processing connections", so a pool opened before lifespan.startup.complete is ready for the first request. Shutdown comes "when the server has stopped accepting connections and closed all active connections". Lifespan runs "once per event loop", so with four workers each process opens its own pool.

To share an object with the requests, the application puts it in scope["state"], and the server passes "a shallow copy of the namespace" into each request's scope. That is the empty 'state': {} in the scope dump. The scope_dump.py example shows the other path: when an application rejects the lifespan scope, the server "must continue but not send any lifespan events".

WSGI code runs on a thread beside the event loop, one request at a time

The project also has reports written for WSGI years ago. The introduction promises they can stay: ASGI "is also designed to be a superset of WSGI", and "a threadpool can be used to run the synchronous WSGI applications away from the async event loop". asgiref, the reference library of the ASGI project, provides the wrapper WsgiToAsgi, which the router above mounts under /legacy:

# legacy.py
import threading


def legacy_wsgi_app(environ, start_response):
    thread = threading.current_thread().name
    start_response("200 OK", [("content-type", "text/plain")])
    line = f"WSGI {environ['REQUEST_METHOD']} {environ['PATH_INFO']} in {thread}\n"
    return [line.encode()]


# orders.py
legacy = WsgiToAsgi(legacy_wsgi_app)

   0.0 ms ** http scope GET /legacy/orders port 55986 in MainThread
   0.3 ms <- http.request body=b'' more_body=False
   1.8 ms -> http.response.start status=200 headers=[(b'content-type', b'text/plain')]
   2.2 ms -> http.response.body body=b'WSGI GET /legacy/orders in ThreadPoolExecutor-0_0\n' more_body=True
   2.4 ms -> http.response.body

The ASGI call ran on MainThread, where the event loop runs. The WSGI function ran on ThreadPoolExecutor-0_0, so a blocking call inside it does not stop the loop. The wrapper read the whole body first, then turned the returned list into one http.response.body with more_body=True and a final empty one.

Then a report that takes one second, with time.sleep(1) standing in for a blocking database query, served alone through WsgiToAsgi. Three browsers ask for it at the same moment:

# slow_legacy.py
import threading
import time

from asgiref.wsgi import WsgiToAsgi


def slow_report(environ, start_response):
    started = time.monotonic()
    time.sleep(1)  # a blocking database query
    thread = threading.current_thread().name
    start_response("200 OK", [("content-type", "text/plain")])
    return [f"{thread} started at {started:.2f}\n".encode()]


app = WsgiToAsgi(slow_report)

request 1: done at 1.03 s, ThreadPoolExecutor-0_0 started at 92103.07
request 2: done at 2.04 s, ThreadPoolExecutor-0_0 started at 92104.08
request 3: done at 3.05 s, ThreadPoolExecutor-0_0 started at 92105.08
all three: 3.05 s

The requests ran one after the other, on the same thread. The cause is in asgiref's source. wsgi.py runs the WSGI application through @sync_to_async with its defaults, and sync.py sets thread_sensitive=True by default. In an async program, its docstring explains, thread-sensitive code runs in "a dedicated single sub-thread that all sync code runs in, one after the other". The default exists for code that is not thread-safe, and the docstring gives SQLite connections as an example. The "threadpool" of the introduction is real, but a pool of one thread.

Django's own ASGI handler avoids this. Its source opens an async with ThreadSensitiveContext(): block around each request, so each request gets its own sync thread. The same three lines of wrapping work for the report:

# slow_legacy_ctx.py
from asgiref.sync import ThreadSensitiveContext

from slow_legacy import app as wrapped


async def app(scope, receive, send):
    async with ThreadSensitiveContext():
        await wrapped(scope, receive, send)

request 2: done at 1.06 s, ThreadPoolExecutor-3_0 started at 92130.82
request 3: done at 1.06 s, ThreadPoolExecutor-4_0 started at 92130.82
request 1: done at 1.06 s, ThreadPoolExecutor-2_0 started at 92130.82
all three: 1.06 s

Figure: Three requests reach a WSGI report that blocks for one second. Under WsgiToAsgi's defaults they share one thread and finish at 3.05 s. With one ThreadSensitiveContext per request, each gets its own thread, and all three finish at 1.06 s. (interactive version in the post)

The context only gives each request a thread. It does not make the WSGI code safe to run in parallel. A report that shares a database connection between threads, or keeps state in globals, was safe only because of the single thread. If the report already ran on a WSGI server with several threads, it is safe here too. If it has only ever run on one thread, keep the default, or keep serving it with a WSGI server.

Async was never the missing piece; a way in for later events was

Back to the title. An async WSGI would still receive everything in one environ at the start and give everything back in one return value. The order page needed three things that shape has no slot for, and the traces showed each one:

Events after the call starts. The pieces of a chunked body, the frames of a WebSocket, and the http.disconnect of a browser that gave up all arrived through receive() while the call was running.
Messages with no request behind them. The websocket.send at 2.2 ms came from a queue, not from the browser.
A start and an end on the event loop. Lifespan let the application set up on the running loop before the server accepted a connection, and close down after the last one closed.

async changes how a server waits. receive and send change what an application can hear and say while it waits. That is the reason ASGI could not be a small update to WSGI.

The takeaway fits in one small middleware. When an ASGI application, or the framework above it, behaves in a way you cannot explain, put a tracing middleware like the one above in front of it and read the events. Both surprises in this post showed up in a trace before they had an explanation: the empty http.request that ended a long poll, and the single ThreadPoolExecutor-0_0 that ran every WSGI request.

References

ASGI specification: the introduction; the ASGI specification, version 3.0; the HTTP and WebSocket message format, version 2.5; the lifespan protocol, version 2.0. The source of all four is in the asgiref repository.
WSGI and WebSocket: PEP 3333 (Eby, 2010); RFC 6455 (Fette and Melnikov, 2011).
History: Andrew Godwin, "Inviting feedback on my proposed 'ASGI' spec", Web-SIG, 9 March 2016; the version history at the end of the ASGI specification.
Source code read for this post: asgiref wsgi.py and sync.py (version 3.13.0); Uvicorn h11_impl.py and websockets_impl.py (version 0.54.0); Django's ASGI handler.
Servers and frameworks: the ASGI project's list of implementations, which includes the servers Uvicorn, Hypercorn, Daphne and Granian, and the frameworks Django, Starlette and FastAPI.
On this blog: the WSGI vs ASGI post, for hello-world applications and when to choose each interface, and the concurrency post, for threads and event loops.

ROI logiciel : mesurer les gains sans surestimer les économies

Mon, 02 Nov 2026 00:00:00 GMT

Mots-clés : retour sur investissement, logiciel, coûts, marge, productivité, mesure des gains

Une équipe envisage un logiciel pour automatiser son support client. Dans ce cas fictif, la première année coûte 30 000 €. Le projet promet de libérer du temps et de mieux servir les clients. Mais quelles améliorations deviennent des gains financiers, et lesquelles restent de la capacité disponible ? Nous allons mesurer le retour sur investissement, ou ROI (Return on Investment, retour sur investissement), en comptant les coûts et les gains attribuables au logiciel. Nous comparerons aussi le point de vue du client avec celui du fournisseur.

Le même logiciel a un ROI différent pour le client et le fournisseur

L’équipe achète un outil, mais son éditeur vend un produit. Leurs calculs portent sur des investissements différents. Un prestataire qui déploie cet outil pour ses clients doit encore choisir un autre périmètre.

Acteur	Décision évaluée	Gains à examiner	Coûts à examiner
Client	Acheter ou développer un outil	Dépenses évitées, marge supplémentaire	Achat, intégration, ressources internes, exploitation
Éditeur	Investir dans un produit ou une fonctionnalité	Revenus supplémentaires attribuables au projet	Développement, acquisition, hébergement, support
Prestataire	Créer une offre ou améliorer sa production	Prestations supplémentaires, coûts évités	Outillage, formation, réalisation, maintenance

L’abonnement payé par le client est un coût pour lui et un revenu pour l’éditeur. Une marge élevée par abonnement ne prouve pas que le développement du produit a déjà été rentabilisé. Il faut aussi couvrir l’investissement et les autres coûts du périmètre choisi.

Pour l’article, nous suivrons le ROI du client sur les douze premiers mois. Le fournisseur pourra reprendre la méthode avec ses propres flux. Cette distinction de points de vue est notre cadrage du cas, pas une norme comptable.

La formule exige une période et un scénario de référence

Nous utiliserons un ROI simple de projet, sans actualisation :

ROI sur la période = (B − C) / C × 100

B = gains financiers supplémentaires attribuables au changement
C = coûts supplémentaires nécessaires pour obtenir ces gains

Les coûts doivent être positifs pour calculer ce ratio. Un gain de 45 000 € pour un coût de 30 000 € laisse 15 000 € nets : le ROI est de 50 %. Les 45 000 € sont les gains avant déduction des coûts du projet. Les appeler « bénéfice net » puis retirer encore ces coûts les compterait deux fois.

Le manuel OpenStax présente ce principe du rendement net rapporté à l’investissement. Notre formule précise son application au projet. Un ROI de 50 % sur douze mois ne signifie pas 50 % par an pour toujours.

L’équipe doit ensuite décrire le scénario de référence. Ici, elle conserve son outil actuel et sous-traite une partie du support. Le coût et les résultats de cette option servent de comparaison. Le Green Book formalise cette comparaison avec la poursuite de l’existant. Ses règles d’évaluation publique ne fixent pas notre seuil de rentabilité d’entreprise.

Avant de remplir la formule, écrivons donc : « Comparer l’achat à la poursuite de l’existant, du lancement à la fin du douzième mois, du point de vue du client. » Cette phrase empêche de comparer des gains sur trois ans à des coûts sur un seul an.

Le coût dépasse l’achat ou le développement initial

La licence ne représente qu’une partie du changement. Pour notre équipe, le budget fictif de la première année est le suivant :

Poste supplémentaire par rapport à l’existant	Montant
Intégration	6 000 €
Migration des données	2 000 €
Formation et temps de démarrage	2 000 €
Abonnement annuel	12 000 €
Usage et hébergement supplémentaires	4 000 €
Maintenance et support supplémentaires	4 000 €
Total sur douze mois	30 000 €

Les trois premiers postes totalisent 10 000 € au démarrage. Les trois suivants représentent 20 000 € sur l’année. Nous supposons que le tarif d’abonnement n’inclut pas les postes facturés séparément. Dans un contrat réel, il faut vérifier cette frontière.

Le coût de formation inclut les ressources internes mobilisées. Même avec des salaires inchangés, leur temps a un coût d’opportunité. Ce budget économique ne décrit donc pas uniquement des sorties de trésorerie supplémentaires. L’article sur les concepts économiques, en espagnol, explique cette distinction et celle entre résultat et trésorerie.

Pour un développement interne, ajouter le travail de conception, de test, de sécurité et d’exploitation nécessaire au changement. Pour une prestation, inclure le pilotage et les reprises prévues au contrat. Prévoir aussi les coûts de sortie si l’horizon couvre le remplacement de l’outil.

Les dépenses déjà engagées et irréversibles ne changent pas entre les options. Les factures futures évitables, elles, comptent. Déclarer les règles d’allocation des coûts partagés rend le calcul vérifiable. Le cadre de la FinOps Foundation aide à relier ces dépenses aux unités métier, comme le coût par dossier résolu.

Les gains reposent sur la marge et les économies réellement obtenues

Le pilote de notre cas fictif suggère 600 heures de support libérées par an. Multiplier ces heures par un taux salarial donne une valorisation de capacité. Cela ne prouve pas que l’entreprise dépensera moins.

Le Government Efficiency Framework distingue précisément ces deux résultats. L’équipe peut traiter davantage de demandes avec les mêmes salaires, sans réduction de sa masse salariale. Ce gain peut être utile ; il faut indiquer comment il sera utilisé.

L’équipe répartit les 600 heures entre deux usages distincts :

200 heures remplacent de la sous-traitance évitable, facturée 75 € l’heure. L’économie prévue est de 15 000 €. Elle exige une baisse réelle des achats, sans minimum contractuel qui maintiendrait la facture.
400 heures permettent des prestations supplémentaires de support payant. Chaque heure vendue apporte 75 € de marge après les coûts variables de réalisation. La marge supplémentaire prévue est de 30 000 €.

Les 75 € de marge du second poste sont déjà nets des coûts variables concernés. Ils excluent les coûts du logiciel, comptés dans les 30 000 € du projet. Nous supposons une demande suffisante pour vendre ces prestations et une qualité de service maintenue. Sans ventes supplémentaires, cette marge reste une prévision.

Le schéma montre où les heures deviennent des gains financiers. La somme 200 + 400 épuise les 600 heures disponibles : aucune heure n’est réutilisée dans les deux branches.

Figure: Les 45 000 € de gains supposent deux usages distincts des 600 heures libérées et la réalisation de chacun. (interactive version in the post)

Une heure libérée doit avoir un usage avant d’avoir un gain financier. Ajouter une « économie salariale » pour ces mêmes 600 heures compterait à nouveau leur valeur, alors que les salaires restent inchangés. Une meilleure satisfaction client peut aussi compter dans la décision. Elle restera un résultat séparé tant que sa traduction financière n’est pas étayée.

Une amélioration doit pouvoir être attribuée au logiciel

Le tableau financier ne prouve pas que le logiciel produit les gains annoncés. Une baisse des tickets peut venir d’un changement de clientèle. Une hausse des prestations peut venir d’une campagne commerciale.

Avant le déploiement, l’équipe définit une période d’observation et des mesures :

Temps de traitement par dossier, à complexité comparable.
Dépenses de sous-traitance réellement évitées.
Volume supplémentaire vendu et marge correspondante.
Qualité : réouvertures, erreurs et satisfaction des clients.

Quand c’est possible, répartir aléatoirement des groupes comparables entre l’ancien et le nouveau fonctionnement. Un test A/B aide à isoler l’effet du changement. Les chercheurs de Microsoft Research décrivent ce principe pour les produits numériques. Il faut aussi vérifier que les groupes ne se contaminent pas et que l’échantillon permet une conclusion utile.

Certains déploiements ne permettent pas cette expérience. Une comparaison avec une équipe témoin ou un déploiement progressif peut aider, avec ses limites. Un simple avant/après reste vulnérable aux changements de volume, de saison ou d’organisation. Le billet sur le raisonnement expérimental en logiciel, en espagnol, explique pourquoi documenter ces autres explications.

Le pilote mesure un effet opérationnel dans certaines conditions. Le ROI prévisionnel extrapole cet effet sur douze mois. Le ROI réalisé utilise les coûts et les gains observés sur la période écoulée. Il faut conserver ces trois niveaux séparés : une extrapolation ne devient pas une mesure parce qu’elle apparaît dans un tableau.

Le cas chiffré révèle les hypothèses qui font basculer le ROI

Sous les hypothèses centrales, les gains atteignent 45 000 € et les coûts 30 000 €. Le résultat économique net est de 15 000 €, soit un ROI simple de 50 % sur douze mois. Il ne s’agit pas d’une mesure de rentabilité observée dans une entreprise.

Voici le calcul exécuté en Python. Les scénarios font varier les gains retenus et les coûts. Ils illustrent des hypothèses, sans leur attribuer de probabilité.

planned_benefits = 200 * 75 + 400 * 75
scenarios = [
    ("Prudent", 0.40, 36_000),
    ("Central", 1.00, 30_000),
    ("Favorable", 1.20, 30_000),
]

for name, realization, costs in scenarios:
    benefits = planned_benefits * realization
    net = benefits - costs
    roi = net / costs * 100
    print(f"{name}: gains={benefits:.0f} EUR, "
          f"coûts={costs:.0f} EUR, net={net:.0f} EUR, ROI={roi:.0f} %")

break_even = 30_000 / planned_benefits * 100
print(f"Part des gains prévus nécessaire pour couvrir les coûts: "
      f"{break_even:.1f} %")

Sortie obtenue :

Prudent: gains=18000 EUR, coûts=36000 EUR, net=-18000 EUR, ROI=-50 %
Central: gains=45000 EUR, coûts=30000 EUR, net=15000 EUR, ROI=50 %
Favorable: gains=54000 EUR, coûts=30000 EUR, net=24000 EUR, ROI=80 %
Part des gains prévus nécessaire pour couvrir les coûts: 66.7 %

À coût constant de 30 000 €, il faut réaliser environ deux tiers des gains prévus pour couvrir les coûts. Si seules les économies de sous-traitance se réalisent, les gains tombent à 15 000 € et le ROI à −50 %. La demande de prestations supplémentaires devient donc une hypothèse à tester avant de généraliser l’outil.

Le simulateur permet d’explorer cette fragilité. Son taux fait varier les deux postes de gains ensemble ; une analyse réelle doit aussi les tester séparément.

Figure: Avec 30 000 € de coûts, réaliser 100 % des 45 000 € prévus donne un ROI de 50 % ; en réaliser 60 % donne −10 %. (interactive version in the post)

La décision dépend aussi du délai, du risque et des résultats observés

Le ROI décrit un rendement relatif. Il ne dit pas quand l’investissement sera récupéré. Le délai de récupération répond à cette autre question.

Dans une variante entièrement exprimée en trésorerie, supposons que les 10 000 € de démarrage soient décaissés au lancement. Les 20 000 € récurrents et les 45 000 € de gains sont payés et encaissés régulièrement pendant douze mois, sans délai ni variation. Le flux net mensuel est alors (45 000 − 20 000) / 12, soit environ 2 083 €. Le délai simple est 10 000 / 2 083, environ 4,8 mois.

Cette estimation exige des flux de trésorerie réels. Elle change si l’abonnement est payé d’avance, si les ventes démarrent tard ou si une partie des coûts valorise du temps interne sans décaissement supplémentaire. Diviser les 30 000 € de coûts annuels par les gains mensuels mélangerait investissement initial et dépenses déjà réparties dans le temps.

Le délai simple ignore les gains après récupération et la valeur temporelle de l’argent, comme le rappelle OpenStax. Sur plusieurs années, examiner aussi la VAN (Valeur actuelle nette : somme des flux futurs ramenés à aujourd’hui, investissement compris.). Le billet sur la valeur temporelle de l’argent, en anglais, en développe le calcul. Un ROI élevé sur un petit projet ne garantit pas davantage de valeur totale qu’un ROI plus faible sur un projet plus grand.

Notre équipe dispose maintenant d’une décision à instruire : valider la demande de prestations, vérifier que la sous-traitance est évitable et mesurer les coûts du pilote. Le scénario central seul ne suffit pas. Aucun pourcentage universel ne remplace ces vérifications.

Avant votre prochain achat ou développement, remplissez cette fiche :

Question	Réponse à écrire avant de lancer le projet
Pour qui ?	Client, éditeur ou prestataire
Par rapport à quoi ?	Option de référence et résultats attendus sans changement
Sur quelle période ?	Dates, montée en charge et durée retenue
À quel coût ?	Postes supplémentaires, ressources internes et règles d’allocation
Pour quels gains ?	Dépenses évitées, marge supplémentaire et capacité séparée
Avec quelles preuves ?	Mesures, comparaison, responsable de chaque gain
Quand réviser ?	Date du bilan, hypothèses à vérifier et critères de poursuite

Commencez par le gain le moins certain : écrivez la preuve qui vous ferait le compter, puis cherchez cette preuve avant de financer le déploiement.

Références

OpenStax, Donna Kirk, 2023. Voir Contemporary Mathematics, Methods of Savings.
HM Treasury, édition 2026. Voir The Green Book.
Ronny Kohavi et ses coauteurs, Microsoft Research, 2009. Voir Online Experimentation at Microsoft.
Government Finance Function et HM Treasury, version 2025. Voir The Government Efficiency Framework.
FinOps Foundation, consulté le 2 novembre 2026. Voir Unit Economics.
OpenStax, Julie Dahlquist et Rainford Knight, 2026. Voir Principles of Finance, Payback Period Method.

Perfect Secrecy and Kerckhoffs' Principle: How a Provably Unbreakable Cipher Was Broken

Mon, 26 Oct 2026 00:00:00 GMT

Keywords: perfect secrecy, one-time pad, Shannon, Kerckhoffs' principle, key reuse, Venona

How do you break a cipher that is mathematically unbreakable? In February 1946, Meredith Gardner, an analyst on the US Army's Venona project, read his first Soviet message protected by a one-time pad. Three years later, Claude Shannon published the proof that a one-time pad, used correctly, cannot be broken, even with unlimited computing power. The question is not only historical: AES (Advanced Encryption Standard) in counter mode and ChaCha20 fail the same way when a key and nonce are reused. This post proves what perfect secrecy requires, states every hypothesis behind the proofs, and shows which one Venona broke.

The path goes from Shannon's definition to the one-time pad and the theorems that set its price. Then it shows why Kerckhoffs' principle, written in 1883, is already part of that definition. The Python examples run in your browser.

Venona read Soviet traffic that was protected by one-time pads

Soviet services encrypted in two steps. A codebook turned words into groups of digits. Then a clerk added key digits, taken from a page of a one-time pad, to those groups. Each page was meant to be used for one message only.

In 1942, under pressure from the German advance on Moscow, the organization that produced the KGB (Komitet Gosudarstvennoy Bezopasnosti: the Soviet Committee for State Security)'s pads printed about 35,000 pages of duplicate key. The copies were sent to users far apart from each other, which made the reuse harder to spot.

On 1 February 1943, a small US Army team led by Gene Grabeel started working on the traffic. Gardner read the first message three years later. The project ran until 1 October 1980. Of the many thousands of messages sent between 1940 and 1948, about 3,000 were at least partly read. They exposed agents such as Klaus Fuchs, Julius Rosenberg and members of the Cambridge Five. The translations became public in 1995.

Figure: Theory and practice of the one-time pad. A Soviet proof that the pad is secure predates the duplicate pages by a year, and Shannon's classified proof predates the first Venona break by a year. The proofs were never wrong. (interactive version in the post)

The mathematics was never broken. Each page, taken alone, was as secure as Shannon's proof says. So why is a duplicated page fatal? To answer that, we need the exact definition, including the hypotheses that usually stay implicit.

Perfect secrecy means the ciphertext does not change the attacker's odds

An encryption scheme has three algorithms. Gen picks a key k from a key space K. Enc turns a message m from the message space M into a ciphertext c from the ciphertext space C. Dec turns c back into m.

Every result in this post rests on the following hypotheses. I will refer to them by name.

H1, finite spaces. M, K and C are finite. K contains only keys that Gen can output, and C contains only ciphertexts that can occur.
H2, independent key. The key is chosen independently of the message.
H3, correctness. Dec is deterministic and Dec_k(Enc_k(m)) = m for every key k and every message m.
H4, known system. The attacker knows Gen, Enc, Dec and the probability of each message. Only the value of k is secret. This is Kerckhoffs' principle, and it gets its own section below.
H5, one message per key. The attacker sees a single ciphertext produced with the key.

The attacker's computing power is unlimited. With that setting, here is Shannon's definition (1949), in the modern form used by Katz and Lindell:

Definition (perfect secrecy). Under H1–H5, a scheme is perfectly secret if for every probability distribution over M, every message m ∈ M and every ciphertext c ∈ C with Pr[C = c] > 0, we have Pr[M = m | C = c] = Pr[M = m].

The right side is what the attacker believes about the message before intercepting anything. The left side is what they believe after seeing c. Perfect secrecy says the two are equal: the ciphertext teaches nothing. The words "every probability distribution" matter. The attacker may already know that "attack at dawn" is far more likely than "retreat", and the scheme must still not add to that knowledge.

There is an equivalent form that is easier to check. Under H1 and H2, a scheme is perfectly secret if and only if, for every pair of messages m0, m1 ∈ M and every ciphertext c ∈ C, Pr[Enc_K(m0) = c] = Pr[Enc_K(m1) = c]. The probability is over the key and over any randomness inside Enc. Every message produces every ciphertext with the same probability, so c carries no trace of which message it came from. The two forms are linked by Bayes' theorem.

The one-time pad is perfectly secret, and the proof takes one line

In the one-time pad, messages, keys and ciphertexts are all strings of n bits. Gen picks k uniformly at random from {0,1}^n. Encryption is Enc_k(m) = m ⊕ k and decryption is Dec_k(c) = c ⊕ k, where ⊕ is bitwise XOR (Exclusive OR: 1 when the two bits differ, 0 when they are equal). Gilbert Vernam built this cipher for teleprinters in 1917, and Joseph Mauborgne added the requirement that the key tape be random and never reused. Steven Bellovin later found that Frank Miller had described the same idea for telegraph codes in 1882.

Theorem (one-time pad). Assume H2 and H5, that messages are n-bit strings, and that the key is uniform over {0,1}^n. Then the one-time pad is perfectly secret.

Proof. Fix a message m and a ciphertext c. The event Enc_K(m) = c happens exactly when K = m ⊕ c, which is one specific key. The key is uniform and independent of m, so Pr[Enc_K(m) = c] = 2^-n for every m and every c. That probability does not depend on m, which is the equivalent form of the definition. ∎

The definition can also be checked directly. The next example enumerates every case exactly, with fractions instead of floats, for n = 2 and a deliberately uneven prior. It compares the one-time pad with a weaker scheme that uses a 1-bit key twice, so its only keys are 00 and 11.

from fractions import Fraction
from itertools import product

MESSAGES = ["00", "01", "10", "11"]
# Any prior works; this one is deliberately uneven.
PRIOR = {"00": Fraction(1, 2), "01": Fraction(1, 4),
         "10": Fraction(1, 8), "11": Fraction(1, 8)}


def xor(a: str, b: str) -> str:
    return "".join("0" if x == y else "1" for x, y in zip(a, b))


def posterior(keys: list[str], c: str) -> dict[str, Fraction]:
    """Pr[M = m | C = c]: uniform key, chosen independently of m."""
    joint = {
        m: PRIOR[m] * Fraction(sum(xor(m, k) == c for k in keys), len(keys))
        for m in MESSAGES
    }
    total = sum(joint.values())
    return {m: p / total for m, p in joint.items()}


one_time_pad = ["".join(bits) for bits in product("01", repeat=2)]  # 4 keys
short_key = [bit * 2 for bit in "01"]  # 2 keys: a 1-bit key used twice

print("m    prior   OTP     short key   (ciphertext c = 00)")
otp, short = posterior(one_time_pad, "00"), posterior(short_key, "00")
for m in MESSAGES:
    print(f"{m}   {str(PRIOR[m]):<7} {str(otp[m]):<7} {short[m]}")

# The OTP check holds for every ciphertext, not just 00.
assert all(posterior(one_time_pad, c) == PRIOR for c in MESSAGES)

m    prior   OTP     short key   (ciphertext c = 00)
00   1/2     1/2     4/5
01   1/4     1/4     0
10   1/8     1/8     0
11   1/8     1/8     1/5

With the one-time pad, the attacker's odds after seeing 00 are the prior, exactly. With the short key, seeing 00 rules out two messages and raises the probability of 00 from 1/2 to 4/5. The attacker did not break anything. They only read the odds that the scheme handed them.

Figure: Each cell holds the key that turns message m (row) into ciphertext c (column). In the one-time pad every cell is filled and each key appears once per row and once per column. With the short key, half the cells are empty: seeing c = 00 leaves only 00 and 11. (interactive version in the post)

Perfect secrecy requires at least as many keys as messages

The short key did not fail by accident. Its failure is a theorem.

Theorem 1 (key-space bound, Shannon 1949). Assume H1 and H3, and let the scheme be perfectly secret. Then |K| ≥ |M|.

The theorem does not assume that Enc is deterministic or that keys are uniform. It only needs correctness and the definition.

Proof. Suppose |K| < |M|. Take the uniform distribution over M and any ciphertext c with Pr[C = c] > 0. Decrypting c with every key gives the set of messages that c could come from. By H3 that set has at most |K| elements, so some message m' is missing from it. If some key could encrypt m' into c, decrypting c with that key would give m', so no key can. Then Pr[M = m' | C = c] = 0, while Pr[M = m'] = 1/|M| > 0. That contradicts perfect secrecy. ∎

In bits: if every n-bit string is a possible message and keys are ℓ-bit strings, then 2^ℓ ≥ 2^n, so ℓ ≥ n. The key must be at least as long as the message. Shannon stated the same bound in 1949: the number of keys must be at least the number of possible messages.

Shannon also characterized the schemes that sit exactly at this bound:

Theorem 2 (Shannon's theorem). Assume H1–H3, that Enc is deterministic, and that |M| = |K| = |C|. The scheme is perfectly secret if and only if both conditions hold:

Gen chooses every key with probability 1/|K|.

For every m ∈ M and every c ∈ C, exactly one key k satisfies Enc_k(m) = c.

Condition 2 is what the left grid in the figure shows: every cell holds exactly one key, and each key appears once in every row and once in every column. The one-time pad is the standard example, and the proof above is the "if" direction of this theorem.

The equal-size hypothesis matters only for the "only if" direction. Without it, a perfectly secret scheme can have extra keys that are not uniform. Take a one-time pad and append one unused bit to the key, chosen with probability 0.9 for 0. The scheme is still perfectly secret, yet it has twice as many keys as messages, the keys are not uniform, and every cell has two keys.

A reused key breaks the one-message hypothesis, not the proof

Now the opening question has an answer. A duplicated page means two messages encrypted with the same key:

c1 ⊕ c2 = (m1 ⊕ k) ⊕ (m2 ⊕ k) = m1 ⊕ m2

The key cancels. The attacker now holds the XOR of two plaintexts, which keeps the structure of language. Theorem 1 explains why this must leak. Encrypting two n-bit messages with one n-bit key is a scheme whose messages are pairs: it has 2^(2n) possible messages and only 2^n keys. By Theorem 1, it cannot be perfectly secret. The short-key scheme from the first example is exactly this case: a 1-bit pad used for two 1-bit messages.

The next example shows the leak in practice with crib dragging. Guess a word that probably appears in one of the messages, slide it along c1 ⊕ c2, and keep the positions where the other message would read as plain text.

import secrets


def xor(a: bytes, b: bytes) -> bytes:
    return bytes(x ^ y for x, y in zip(a, b))


m1 = b"SEND THE REPORT TO THE CONSULATE"
m2 = b"AGENT MEETS COURIER AT THE DOCKS"
key = secrets.token_bytes(len(m1))  # perfect key... used twice

c1, c2 = xor(m1, key), xor(m2, key)
leak = xor(c1, c2)
assert leak == xor(m1, m2)  # the key has cancelled out

# Crib dragging: guess a word that is probably in one message,
# slide it along the leak, and keep positions where the other
# message would read as plain uppercase text.
crib = b" THE "
for i in range(len(leak) - len(crib) + 1):
    other = xor(leak[i : i + len(crib)], crib)
    if all(ch == 32 or 65 <= ch <= 90 for ch in other):
        print(i, other.decode())

4 T MEE
11 PXUDR
18 R AT
22  CONS

Three of the four hits are real fragments. T MEE and R AT come from m2, where m1 holds " THE ". CONS comes from m1, where m2 holds " THE ". The attacker does not know which message contains the guess, and does not need to. PXUDR is noise that a human discards at a glance. Each confirmed fragment suggests the next guess, and the two messages come out together.

Run the example twice. The key is new each time, and the output is identical. That is what "the key cancels" means.

Venona worked the same way. Soviet clerks added digits instead of XORing bits, so the analysts subtracted, and the key cancelled just the same. Their task was harder, because they also had to rebuild the codebook before any of it read as words. But the opening that made it possible was the reused key.

The theory was not the problem. According to a 2006 account by Sergei Molotkov, the Soviet engineer Vladimir Kotelnikov had proved the absolute security of the one-time pad in a 1941 report that apparently remains classified. A year later, the same state printed duplicate pages. The proof held. Production broke one of its hypotheses.

Reuse is not the only way out of the theorem. Each hypothesis has a failure mode of its own:

A key that is not uniform (H2 and the theorem's uniform key). If a 256-bit pad is generated from a 32-bit seed, there are only 2^32 real keys. By Theorem 1, that is far too few for 256-bit messages.
Messages of different lengths (H1). The definition fixes M, and the one-time pad uses messages of one length n. A ciphertext reveals the length of its message unless messages are padded to a common length.
Tampering (not covered at all). Flipping bit i of c flips bit i of m. Perfect secrecy is about what the attacker learns, not about what they can change. Integrity needs a separate mechanism, such as a message authentication code.

Kerckhoffs' principle is built into the definition

In 1883, Auguste Kerckhoffs published La cryptographie militaire in the Journal des sciences militaires, in two parts, in January and February. It lists six requirements for military field ciphers. The second one is the principle that carries his name:

"Il faut qu'il n'exige pas le secret, et qu'il puisse sans inconvénient tomber entre les mains de l'ennemi."

The system must not require secrecy, and it must be able to fall into the hands of the enemy without inconvenience.

Shannon adopted the same assumption in 1949, and it is often quoted as Shannon's maxim: "the enemy knows the system." In this post it is hypothesis H4. The definition of perfect secrecy takes probabilities over the key only. The attacker knows Gen, Enc, Dec and the prior over messages. Nothing else is assumed secret, so the definition measures the strength of the key and nothing else.

Why build that into the definition? Because a secret algorithm is key material with bad properties. It is copied into every device that runs it, so it leaks. It cannot be changed quickly once it leaks. And it cannot be reviewed in public before it is deployed. Two well-documented cases:

GSM's A5/1. The cipher that protected GSM (Global System for Mobile Communications) calls was kept secret. Its general design leaked in 1994, and Marc Briceno reverse-engineered it in full from a GSM phone in 1999. Alex Biryukov, Adi Shamir and David Wagner presented a real-time attack in 2000.
MIFARE Classic's Crypto-1. NXP's contactless cards used a proprietary cipher with a 48-bit key. In 2008, Karsten Nohl, David Evans, Starbug and Henryk Plötz reverse-engineered it from images of the chip's silicon. The same year, Flavio Garcia and colleagues at Radboud University recovered it from card-reader traffic and published practical key-recovery attacks.

Kerckhoffs' principle does not require publishing the design. It requires that publication cost nothing.

Kerckhoffs also accepted that perfect secrecy is too expensive

His first requirement is less famous and just as relevant:

"Le système doit être matériellement, sinon mathématiquement, indéchiffrable."

The system must be practically, if not mathematically, undecipherable.

Kerckhoffs settled for practical security 66 years before Theorem 1 proved why. Mathematical security needs a key at least as long as all the traffic it protects, generated at random, delivered in advance and never reused.

Modern ciphers take the practical branch. AES uses a public design and a 128- or 256-bit key to encrypt gigabytes. By Theorem 1, it cannot be perfectly secret for any message longer than its key. Its security is computational: the best known attacks would take an infeasible amount of work. That is an assumption supported by decades of public analysis, not a theorem.

Stream constructions such as AES in counter mode and ChaCha20 then imitate the pad. They expand the key and a nonce, a value that must be used only once per key, into a long keystream, and XOR it with the message. Reuse a key-nonce pair and the result is the two-time pad from the example above.

Hypothesis	Where it is used	What breaks if it fails	Example
Key uniform and independent (H2)	One-time pad theorem	The ciphertext shifts the attacker's odds	A 256-bit pad generated from a 32-bit seed
At least as many keys as messages	Theorem 1	Some messages are ruled out	The short-key scheme in the first example
One message per key (H5)	The definition	`c1 ⊕ c2 = m1 ⊕ m2`: the key cancels	Venona; a reused key-nonce pair in AES-CTR
Only the key is secret (H4)	The definition, Kerckhoffs	Security rests on a secret that leaks	A5/1, Crypto-1
Messages of one length (H1)	The one-time pad's `M`	The ciphertext reveals the message length	Unpadded messages of different sizes

A provably unbreakable cipher breaks when one of its hypotheses fails

How do you break a cipher that is mathematically unbreakable? You do not attack the proof. You find the hypothesis that the deployed system no longer meets. In Venona, it was a single one: each key page had to be used once, and 35,000 pages were not. Shannon's theorems and Kerckhoffs' principle describe the same boundary from two sides: the math protects you only as far as its hypotheses go, and the key is the only secret it allows.

The takeaway fits in one habit. When someone says a system is unbreakable, ask for the theorem and read its hypotheses. In code, that habit becomes three rules. Take keys from a cryptographically secure generator, such as Python's secrets module. Never encrypt twice with the same key and nonce. Design as if the attacker already has your source code.

References

Shannon, C. E. (1949). Communication Theory of Secrecy Systems. Bell System Technical Journal, 28(4), 656–715. An earlier version circulated as the classified Bell Labs memorandum A Mathematical Theory of Cryptography (MM 45-110-02, 1 September 1945).
Kerckhoffs, A. (1883). La cryptographie militaire. Journal des sciences militaires, IX, 5–38 (January) and 161–191 (February). Transcription and scans by Fabien Petitcolas.
Katz, J., & Lindell, Y. Introduction to Modern Cryptography, chapter 2 (perfectly secret encryption). CRC Press.
National Security Agency. VENONA: An Overview. PDF.
Vernam, G. S. U.S. Patent 1,310,719, Secret Signaling System, issued 22 July 1919.
Bellovin, S. M. (2011). Frank Miller: Inventor of the One-Time Pad. Cryptologia, 35.
Molotkov, S. N. (2006). Quantum cryptography and V. A. Kotel'nikov's one-time key and sampling theorems. Physics-Uspekhi, 49(7).
Biryukov, A., Shamir, A., & Wagner, D. Real Time Cryptanalysis of A5/1 on a PC. Fast Software Encryption (FSE 2000), LNCS, Springer, 2001.
Nohl, K., Evans, D., Starbug, & Plötz, H. (2008). Reverse-Engineering a Cryptographic RFID (Radio-Frequency Identification) Tag. USENIX Security Symposium.
Garcia, F. D., de Koning Gans, G., Muijrers, R., van Rossum, P., Verdult, R., Wichers Schreur, R., & Jacobs, B. (2008). Dismantling MIFARE Classic. ESORICS 2008.

Análisis de datos: los cuatro tipos y qué información usa cada uno

Wed, 21 Oct 2026 00:00:00 GMT

Palabras clave: análisis descriptivo, análisis predictivo, análisis diagnóstico, análisis prescriptivo, prueba A/B, optimización

El análisis de datos suele dividirse en cuatro tipos, y cada uno responde una pregunta distinta: qué está sucediendo, qué va a suceder, por qué sucede y qué debo hacer. Lo que más los diferencia no es la herramienta, sino la información que necesitan y lo que entregan a la fase siguiente.

Para que las diferencias sean concretas, este artículo sigue un solo problema de principio a fin. Una tienda en línea con 1900 clientes activos quiere que menos clientes dejen de comprar, lo que se conoce como abandono o churn. Con el mismo presupuesto de $2500 en cupones, repartirlos a quienes tienen más riesgo deja $2600 de beneficio neto, y combinar las cuatro fases deja $5600. La diferencia está en qué información usa cada fase.

Figure: Una tienda en línea que quiere perder menos clientes recorre las cuatro fases. Cada salida alimenta la fase siguiente, y la decisión final produce los datos con los que empieza la próxima vuelta. (interactive version in the post)

1. Análisis descriptivo: ¿qué está sucediendo?

Qué información usa. Los datos que la organización ya tiene: pedidos, visitas, registros de soporte, logs y transacciones. Casi siempre son tablas históricas, aunque también pueden ser eventos, texto o series de tiempo. No hace falta etiquetar nada ni generar datos nuevos.

Cómo la usa.

Limpia y perfila los datos: valores nulos, duplicados, tipos y rangos imposibles. Un análisis descriptivo sobre datos sucios describe los errores de captura, no el negocio.
Hace análisis exploratorio de datos (EDA): distribuciones, medias y medianas, valores atípicos, agrupaciones por segmento y tendencias en el tiempo.
Calcula correlaciones entre variables (Pearson para relaciones lineales, Spearman para relaciones monótonas) para ver qué se mueve junto con qué.
Comunica el resultado con tableros, reportes y gráficos.

En la tienda, este análisis muestra que el 28 % de los clientes del trimestre pasado no volvió a comprar en los 90 días siguientes. La cifra sube al 40 % entre quienes registraron una queja y al 60 % entre quienes llevan más de 60 días sin comprar. Los días desde la última compra y el número de quejas se mueven junto con el abandono.

Buscamos responder a la pregunta "¿qué está sucediendo y por qué?", y la segunda mitad merece cuidado: una correlación sugiere un "por qué", pero no lo demuestra. Que los clientes con quejas se vayan más puede deberse a la queja, a un producto defectuoso que causó ambas cosas o a cualquier otra variable que no medimos. El descriptivo produce hipótesis, y las fases siguientes las ponen a prueba. Si te interesa la frontera entre describir una muestra y sacar conclusiones sobre una población, la explico en estadística descriptiva e inferencial.

2. Análisis predictivo: ¿qué va a suceder?

Qué información usa. Un histórico etiquetado: cada fila es una observación pasada con sus variables de entrada (features) y el resultado que ocurrió después (target). En la tienda, cada fila es un cliente en una fecha. Sus variables son los días desde la última compra, el número de pedidos, el ticket promedio y las quejas, y su etiqueta dice si dejó de comprar en los 90 días siguientes. La calidad, la cantidad y la representatividad de esos datos pesan más que la elección del algoritmo.

Cómo la usa.

Regresión cuando el resultado es un número: las ventas del próximo mes o el margen que dejará un cliente durante el año.
Clasificación cuando el resultado es una categoría: se va o se queda, fraude o no fraude.
Pronóstico cuando el resultado es una serie de tiempo: la demanda de cada semana.

El flujo típico es dividir los datos en entrenamiento, validación y prueba, entrenar el modelo, medirlo con métricas como RMSE (Root Mean Square Error (raíz del error cuadrático medio)), precisión, recall o AUC (Area Under the ROC Curve (área bajo la curva ROC)), y desplegarlo para puntuar datos nuevos. Dos detalles importan mucho cuando la predicción alimenta decisiones. Primero, la división debe respetar el tiempo: entrenar con trimestres antiguos y evaluar con el más reciente, y no usar variables que se miden después del resultado (data leakage). Segundo, las probabilidades deben estar calibradas: si el modelo dice 40 %, cerca de 4 de cada 10 de esos clientes deberían irse. La fase prescriptiva usará esa cifra como probabilidad, no solo como ranking.

El resultado en la tienda es un riesgo por cliente: la probabilidad de que se vaya en los próximos 90 días. Pero predecir no es explicar. El modelo puede acertar usando variables que el negocio no controla, y aprende asociaciones del pasado: no dice qué pasaría si la tienda cambiara algo. Esa pregunta es la de la fase siguiente.

3. Análisis diagnóstico: ¿por qué sucedió esto?

Qué información usa. Además de los datos históricos, información generada a propósito para aislar una causa: un grupo de control y uno de tratamiento, una asignación al azar entre ambos y una sola variable que cambia. También usa las hipótesis que dejó el descriptivo y el contexto de lo que pasó en el periodo, como campañas, cambios de precio o fallas del sitio.

Cómo la usa.

Convierte una hipótesis en algo que se puede refutar: "un cupón del 15 % reduce el abandono".
Diseña un experimento o prueba A/B: reparte a los clientes al azar, da el cupón solo a uno de los grupos y mide la diferencia. El azar es lo que permite hablar de causa, porque reparte por igual entre los dos grupos todo lo que no medimos.
Decide el tamaño de la muestra antes de empezar, con un cálculo de potencia, y no mira los resultados a mitad de camino para detenerse cuando le convienen.
Analiza la diferencia con estadística inferencial: intervalos de confianza, valores p y tamaño del efecto.

En la tienda, el trimestre anterior se repartió al azar a 1800 clientes: 900 recibieron el cupón y 900 no.

Figure: La prueba A/B del cupón. Sin cupón se fue el 28 % del grupo; con cupón, el 22 %. La diferencia es de 6 puntos porcentuales y su intervalo de confianza del 95 %, de 2 a 10 puntos, no incluye el cero, así que el efecto no se explica solo por el azar. (interactive version in the post)

El cálculo cabe en unas pocas líneas, usando la aproximación normal para la diferencia de dos proporciones:

from math import sqrt

n = 900
p_control, p_cupon = 252 / n, 198 / n  # 0.28 y 0.22
efecto = p_control - p_cupon  # 0.06: 6 puntos porcentuales
error = sqrt(p_control * (1 - p_control) / n + p_cupon * (1 - p_cupon) / n)
bajo, alto = efecto - 1.96 * error, efecto + 1.96 * error  # 0.020 y 0.100

Cuando no se puede experimentar, por costo, por ética o porque el hecho ya ocurrió, se usan métodos de inferencia causal sobre datos observacionales, como diferencias en diferencias, regresión discontinua o emparejamiento. Sus conclusiones dependen de supuestos más fuertes que los de un experimento.

Hay un matiz que la fase siguiente necesita: el efecto de 6 puntos es un promedio. El mismo cupón puede retener a muchos clientes que acaban de quejarse y casi a ninguno de los que ya se fueron mentalmente. Medir el efecto por segmento exige experimentos más grandes o modelos de uplift, que estiman cuánto cambia el resultado de cada cliente si se actúa. Con 900 clientes por grupo, los intervalos de cada segmento serían muy anchos. En el ejemplo supongo que esos efectos ya están bien estimados.

4. Análisis prescriptivo: ¿qué debo hacer?

Qué información usa. Las salidas de las fases anteriores más la descripción del problema de decisión:

Predicciones: el riesgo de cada segmento y el margen que dejará cada cliente que se quede, que también es una predicción.
Efectos causales: cuánto cambia el resultado si se actúa, medido en el diagnóstico.
Recursos: presupuesto, personal, inventario o capacidad.
Objetivo: qué se maximiza o minimiza, como el beneficio neto.
Restricciones: límites de capacidad, normas, plazos y costos de cada acción.

Segmento	Clientes	Riesgo (predictivo)	Efecto del cupón (diagnóstico)	Margen por cliente	Neto por cupón
Queja reciente	200	40 %	15 pp	$150	+$17,50
Nuevos sin 2.ª compra	400	35 %	8 pp	$90	+$2,20
Inactivos 60+ días	300	60 %	2 pp	$100	−$3,00
Sensibles al precio	350	25 %	10 pp	$80	+$3,00
Mayoristas	150	15 %	4 pp	$400	+$11,00
Frecuentes satisfechos	500	5 %	0,4 pp	$160	−$4,36

Cada cupón cuesta $5. El neto por cupón es el margen que protege, efecto por margen, menos ese costo: para el segmento con queja reciente, 0,15 × $150 − $5 = $17,50.

Cómo la usa. Formula el problema como una optimización. Si k_i es el número de cupones para el segmento i:

maximizar   Σ k_i · (efecto_i · margen_i − costo)
sujeto a    Σ k_i · costo ≤ presupuesto
            0 ≤ k_i ≤ clientes_i

Es un problema de programación lineal. Como todos los cupones cuestan lo mismo, su solución es sencilla: llenar primero los segmentos con mayor neto por cupón y no dar cupones donde el neto es negativo. Es la regla de la mochila fraccionaria, que toma primero lo que más vale por unidad de peso. Con más tipos de acción, costos distintos o restricciones enteras se usan programación entera, simulación o aprendizaje por refuerzo.

Figure: Mueve el presupuesto y cambia de estrategia. Con $2500, la estrategia óptima retiene a 51 clientes y deja $5600 de beneficio neto; repartir por riesgo retiene a 36 y deja $2600, porque gasta $1500 en clientes inactivos a los que el cupón casi no cambia. (interactive version in the post)

El explorable muestra la lección central de la fase: el riesgo dice quién se va; el efecto dice a quién puedes retener. Los inactivos tienen el riesgo más alto, pero el cupón casi no los mueve. Los frecuentes satisfechos se quedarían de todos modos. Una estrategia que solo usa la predicción gasta el presupuesto en ellos. Fíjate también en que, con presupuesto de sobra, la estrategia óptima deja dinero sin gastar: el objetivo es el beneficio neto, no repartir todos los cupones.

Esta es la fase más valiosa y también la más difícil, porque exige modelar bien tanto los datos como el negocio. Un error en el margen, en el costo o en una restricción produce una decisión precisa y equivocada.

El orden correcto depende de qué se quiere medir

Este artículo recorrió las fases en el orden descriptivo, predictivo, diagnóstico y prescriptivo. El modelo de madurez analítica de Gartner pone el diagnóstico en segundo lugar y el predictivo en tercero. Los dos órdenes son coherentes, porque usan criterios distintos.

Figure: Las mismas cuatro fases en dos órdenes. Gartner va de mirar atrás a mirar adelante. Este artículo va de observar asociaciones a intervenir y, al final, decidir. (interactive version in the post)

Gartner ordena por madurez y valor para el negocio. Primero se entiende el pasado (qué pasó y por qué) y luego se mira al futuro (qué pasará y qué hacer). En ese modelo, el diagnóstico suele ser trabajo sobre los datos que ya existen: desglosar un indicador por segmento, comparar periodos y buscar la causa raíz. Es más fácil que entrenar un modelo, por eso va antes.
Este artículo ordena por lo que exige cada fase. Aquí el diagnóstico se define con experimentos y pruebas A/B, así que el orden sigue una escalera parecida a la de causalidad de Judea Pearl. Primero viene la asociación: observar datos, que es lo que hacen la correlación y la predicción. Luego, la intervención: cambiar algo y medir qué pasa. Al final, la decisión, que necesita saber qué pasa si se interviene. Miguel Hernán y sus coautores proponen una división similar de las tareas de la ciencia de datos: descripción, predicción e inferencia causal.

La elección práctica depende de la pregunta. Para medir la madurez de un equipo o planear qué capacidades construir primero, el orden de Gartner funciona bien. Para decidir qué evidencia necesita una decisión, el orden de este artículo es más preciso: una predicción, por buena que sea, no reemplaza a un experimento cuando la pregunta es qué pasa si actúo.

Cada fase entrega lo que la siguiente necesita

Fase	Pregunta	Información que usa	Cómo la usa	Resultado
Descriptivo	¿Qué está sucediendo?	Datos históricos que ya existen	Limpieza, EDA, correlación, tableros	Línea base e hipótesis
Predictivo	¿Qué va a suceder?	Histórico etiquetado: variables y resultado	Regresión, clasificación, pronóstico	Probabilidades y pronósticos
Diagnóstico	¿Por qué sucedió esto?	Datos generados con grupos asignados al azar	Experimentos, pruebas A/B, inferencia causal	Efectos causales con su margen
Prescriptivo	¿Qué debo hacer?	Predicciones, efectos, recursos, objetivos, límites	Optimización: asignación, decisión, planificación	Una acción concreta

Cada fase depende de lo que entrega la anterior, y la decisión final cierra el ciclo: al ejecutarse, produce los datos que la próxima vuelta del análisis descriptivo va a resumir.

Si te quedas con una sola idea, que sea la de la tienda: la predicción dice quién se va, pero solo el experimento dice a quién puedes retener. Una decisión que use solo la primera gasta el presupuesto donde el cupón no cambia nada.

Método científico aplicado al desarrollo de software y la seguridad

Mon, 19 Oct 2026 00:00:00 GMT

Palabras clave: método científico, falsabilidad, hipótesis, threat modeling, registro de premisas

En seguridad tomamos decisiones apoyadas en supuestos: "este servicio no es alcanzable desde fuera de la VPC (Virtual Private Cloud (nube privada virtual))", "nuestro SAST (Static Application Security Testing (pruebas estáticas de seguridad de aplicaciones)) detecta inyecciones", "el token expira en 15 minutos". Si no los verificamos, pueden convertirse en puntos ciegos. La idea central de este artículo es sencilla: toda premisa es una hipótesis hasta que la sometes a una prueba que podría refutarla.

El ciclo, traducido al trabajo diario

Observación: un comportamiento o dato concreto, como un pico de errores 403, un hallazgo del SAST o un incidente.
Hipótesis falsable: una afirmación que puede resultar falsa. "Este servicio es inalcanzable desde fuera de la VPC" lo es. "Nuestra arquitectura es segura" no.
Predicción: si la hipótesis es cierta, qué debería pasar en una prueba concreta, y qué resultado la refutaría.
Experimento: un test, un fuzzing, un escaneo, un ejercicio de red team o un benchmark, con las variables controladas.
Análisis y registro: se documentan todos los resultados, incluidos los negativos y los que contradicen lo esperado.
Revisión por pares: en la práctica son el code review, la revisión del threat model y el postmortem sin culpables.

El criterio que separa esto de una opinión es la falsabilidad: una afirmación empírica que ningún resultado posible puede contradecir no ofrece un criterio de prueba. Los requisitos normativos y las propiedades demostradas formalmente se evalúan de otras maneras.

Dónde aparece en la práctica

Threat models como conjuntos de hipótesis

Muchos supuestos de un threat model ("el atacante no tiene acceso al bucket", "el token expira en 15 minutos") pueden convertirse en una verificación automática sobre la infraestructura o en un test en el pipeline de CI (Continuous Integration (integración continua)). Así el modelo deja de ser un documento que envejece y pasa a ser una batería de pruebas que puede fallar. El test del inicio de este artículo es exactamente eso: la hipótesis "el token expira a los 15 minutos" escrita de forma que pueda ser refutada.

Medir las herramientas en lugar de creerles

"Nuestro SAST detecta inyecciones" es una afirmación empírica. Para evaluarla puedes sembrar vulnerabilidades conocidas en un corpus de prueba y comprobar cuáles detecta la herramienta y cuáles no. Lo que no detecta es información valiosa sobre sus límites.

Cambios con línea base

Antes de afirmar que una mitigación redujo el riesgo, compara contra un baseline. Una línea base ayuda a interpretar el cambio, pero una comparación antes/después no basta para establecer causalidad: el tiempo medio de remediación pudo bajar porque cambió el equipo, no porque cambió el proceso.

Postmortems y reproducibilidad

Un postmortem sólido formula hipótesis causales y busca evidencia que las desmienta, en lugar de quedarse con la primera explicación plausible. Una prueba de concepto reproducible fortalece la evidencia, pero un incidente también puede establecerse con registros, trazas o evidencia forense aunque no sea posible repetirlo. Hay que documentar la incertidumbre restante.

Un contraejemplo vale más que mil confirmaciones

Una afirmación universal como «este endpoint nunca acepta peticiones sin autenticación» queda refutada por una petición no autenticada que sí funciona. En cambio, una petición rechazada no prueba que ningún otro intento pueda funcionar. El contraejemplo demuestra que esa propiedad concreta falla en las condiciones probadas; no clasifica toda la seguridad del sistema.

Límites que conviene recordar

Una batería finita de pruebas no suele demostrar ausencia de errores. La conclusión debe limitarse a las condiciones evaluadas. Las verificaciones exhaustivas de modelos finitos y las pruebas formales pueden demostrar propiedades dentro de sus modelos y supuestos.
El adversario se adapta. El sistema no es estacionario: lo que fue cierto ayer puede dejar de serlo cuando el atacante cambia de táctica, así que las hipótesis necesitan revalidarse de forma continua.
Goodhart. Cuando una métrica se vuelve objetivo, deja de ser buena métrica. "Número de vulnerabilidades cerradas" es el ejemplo clásico.
Sesgo de confirmación. Es el riesgo principal. La mejor defensa es escribir la hipótesis, la predicción y el criterio de refutación antes de ejecutar la prueba, para no ajustarlos después de ver el resultado.

Un método práctico para empezar: el registro de premisas

Crea una tabla con tus supuestos de seguridad y cuatro columnas:

Afirmación	Impacto si es falsa	Evidencia actual	Resultado que la refutaría
El bucket no es accesible público	Alto	Revisión manual hace 6 m	Leer el bucket sin autorización
El token expira en 15 minutos	Medio	Configuración leída	Obtener acceso con un token vencido
El SAST detecta inyecciones SQL	Alto	Ninguna	Omitir una inyección del corpus de prueba

Prioriza las filas de alto impacto y baja evidencia, y conviértelas en pruebas repetibles que corran de forma continua. Cada vez que una falle, habrás aprendido algo que una creencia sin verificar nunca te habría dicho.

Resumen

El método científico es un marco simple: formular hipótesis falsables, predecir qué debería ocurrir, ponerlas a prueba, registrar todos los resultados y someterlos a revisión. Aplicado a la seguridad, convierte los supuestos implícitos en pruebas repetibles. Lo que importa es el hábito: escribir qué esperas, qué te haría cambiar de opinión, y buscar activamente esa evidencia.

Descriptive vs Inferential Statistics: What Seven Numbers Can Tell You

Sun, 29 Mar 2026 00:00:00 GMT

Take seven measurements: 23, 45, 12, 67, 34, 89 and 21. You can say exactly what they look like. You cannot yet say what the next measurement, or the whole population they came from, will look like. The first job belongs to descriptive statistics. The second belongs to inferential statistics, and mixing them up is how a small sample turns into a confident, wrong conclusion.

Descriptive statistics summarizes the data you have

The goal of descriptive statistics is to summarize a dataset with a small number of measures. Faced with a heap of raw numbers, it asks how we can find the pattern and present it in a digestible way. It does this without assuming anything beyond the dataset itself. The most common summaries are:

Central tendency: the mean, median, and mode identify the center of the data.
Dispersion: the variance, standard deviation, and interquartile range (IQR) describe how stretched or squeezed the data are.
Shape and relationship: skewness, kurtosis, and the linear correlation between two variables.

For our seven measurements, the mean is 41.57 and the standard deviation is 27.71 (dividing by n - 1). Those two numbers describe the sample completely, and they say nothing about anyone outside it.

Inferential statistics estimates what you did not measure

Descriptive statistics describes the exact data on hand. Inferential statistics generalizes from a sample to a larger population. Measuring a whole population, such as every person in a country, is often impossible, slow, or expensive. So researchers measure a sample and use a sampling design and a statistical model to estimate the population parameters. A large sample alone does not remove selection bias.

This works because inferential statistics relies on the laws of probability to reach conclusions and to state how confident we are in them. It uses inductive reasoning: the logic flows from the specific sample up to a broader claim about the population.

The key techniques are:

Point estimation: a single "best guess" for an unknown population parameter.
Interval estimation (confidence intervals): an interval built with a specified long-run coverage rate under the model's assumptions. A 95% procedure covers the fixed population parameter in 95% of repeated samples. It does not assign a 95% probability to that parameter after you observe an interval.
Hypothesis testing: an assessment of how compatible the data are with a null hypothesis and its assumptions. A p-value is not the probability that the hypothesis is true, and statistical significance does not measure effect size.

From seven numbers to a range for the population mean

Here is the step from description to inference. Assume the seven observations are an independent random sample from a normal population. A Student's t interval uses the sample standard deviation (ddof=1), six degrees of freedom, and mean ± 2.447 * sample_std / sqrt(7). With only seven observations, the normality assumption matters. See NIST (National Institute of Standards and Technology)'s confidence limits for the mean.

Figure: The seven values have mean 41.57. Under the stated sampling assumptions, the 95% t confidence interval for the population mean μ is 15.9 to 67.2. The interval reflects both the small sample and its variability. (interactive version in the post)

The sample mean, 41.57, is a description. The range from 15.9 to 67.2 is an inference. It is wide because seven observations carry little information and the data vary a lot. A larger sample or less variable data would narrow it.

Describe first, then infer, and state your assumptions

Descriptive statistics tells you what is in your sample. Inferential statistics uses probability to say what that sample suggests about the population, and how uncertain that suggestion is. Always describe the data before you infer from it. Then check that your sampling and model assumptions hold, because the interval is only as trustworthy as they are.

Classic Algorithms Explained

Sun, 22 Mar 2026 00:00:00 GMT

Let's dive into some of the most important algorithms in computer science. Whether you're preparing for technical interviews, optimizing code, or just curious about how software works under the hood, understanding these classics is essential. We'll explore what each algorithm does, why it matters, and how to implement it.

Karatsuba Multiplication

Multiplying two numbers might seem straightforward—just use the * operator, right? But when you're dealing with very large numbers (thousands or millions of digits), the naive approach becomes inefficient. That's where Karatsuba multiplication comes in.

What It Does

Karatsuba multiplication is a divide-and-conquer algorithm that multiplies two n-digit numbers faster than the grade-school method. Instead of doing a single O(n²) multiplication, it breaks the problem into smaller pieces and recombines them cleverly.

The key insight: to multiply two 2-digit numbers xy and ab, you can use three multiplications instead of four:

Instead of computing all four products from scratch, you reuse intermediate results
This reduces the number of recursive multiplications needed

Time Complexity

Time: O(n^log₂(3)) ≈ O(n^1.585) — much better than O(n²) for large numbers
Space: O(n) for storing intermediate results

Python Implementation

def karatsuba(x, y):
    """
    Multiply two integers using Karatsuba algorithm.

    Args:
        x, y: integers to multiply

    Returns:
        The product x * y
    """
    # Base case: small numbers, use standard multiplication
    if x < 10 or y < 10:
        return x * y

    # Determine the number of digits
    n = max(len(str(x)), len(str(y)))
    m = n // 2

    # Split the numbers
    # x = x_high * 10^m + x_low
    # y = y_high * 10^m + y_low
    x_high, x_low = divmod(x, 10**m)
    y_high, y_low = divmod(y, 10**m)

    # Recursive multiplications
    z0 = karatsuba(x_low, y_low)
    z2 = karatsuba(x_high, y_high)
    z1 = karatsuba(x_high + x_low, y_high + y_low) - z0 - z2

    # Combine results
    return z2 * (10 ** (2 * m)) + z1 * (10 ** m) + z0

# Example usage
print(karatsuba(1234, 5678))  # Output: 7006652

Why It Matters

For cryptography and big integer arithmetic (common in blockchain and scientific computing), Karatsuba provides a significant speedup. Python's native int type uses algorithms like Karatsuba internally for efficient large number multiplication.

Binary Search

If you've ever searched through a sorted dictionary, you were doing binary search in your head. This is one of the most elegant algorithms—simple to understand, powerful in practice.

How Binary Search Works

Binary search finds a target value in a sorted array by repeatedly dividing the search interval in half. At each step, you compare the middle element to your target and eliminate half the remaining elements.

Figure: Searching for 23 among 11 sorted values. Each iteration keeps only the half that can still hold the target, so 4 comparisons are enough. (interactive version in the post)

Binary Search Complexity

Time: O(log n) — incredibly fast even for huge datasets
Space: O(1) for iterative version; O(log n) for recursive (call stack)

Binary Search in Python

def binary_search(arr, target):
    """
    Find target in a sorted array.

    Args:
        arr: sorted list of comparable elements
        target: value to find

    Returns:
        Index of target if found, -1 otherwise
    """
    left, right = 0, len(arr) - 1

    while left <= right:
        mid = (left + right) // 2

        if arr[mid] == target:
            return mid
        elif arr[mid] < target:
            # Target is in the right half
            left = mid + 1
        else:
            # Target is in the left half
            right = mid - 1

    return -1  # Not found

# Example usage
numbers = [1, 3, 5, 7, 9, 11, 13, 15, 17, 19]
print(binary_search(numbers, 7))   # Output: 3
print(binary_search(numbers, 10))  # Output: -1

Why Binary Search Matters

Binary search is the foundation of many database and systems techniques. Any time you need to search a sorted collection—from database indexes to finding the insertion point in a data structure—binary search principles apply. It's O(log n) elegance in action.

Merge Sort

Merge sort is a stable, divide-and-conquer sorting algorithm that consistently delivers O(n log n) performance, regardless of input. It's widely used in practice and forms the basis of timsort (used in Python).

How Merge Sort Works

Merge sort divides an array into single-element arrays (which are trivially sorted), then merges them back together in sorted order. The merge step is where the magic happens—combining two sorted arrays into one sorted array in linear time.

Figure: Eight values split into single elements over log₂ 8 = 3 levels, then merge back in sorted order over 3 more. Every level touches all n values, which is where O(n log n) comes from. (interactive version in the post)

Merge Sort Complexity

Time: O(n log n) in all cases (best, average, worst)
Space: O(n) — requires extra space for merging

Merge Sort in Python

def merge_sort(arr):
    """
    Sort array using merge sort algorithm.

    Args:
        arr: list to sort

    Returns:
        Sorted list
    """
    if len(arr) <= 1:
        return arr

    # Divide
    mid = len(arr) // 2
    left = merge_sort(arr[:mid])
    right = merge_sort(arr[mid:])

    # Conquer (merge)
    return merge(left, right)

def merge(left, right):
    """
    Merge two sorted lists into one sorted list.
    """
    result = []
    i = j = 0

    # Compare elements from left and right, add smaller
    while i < len(left) and j < len(right):
        if left[i] <= right[j]:
            result.append(left[i])
            i += 1
        else:
            result.append(right[j])
            j += 1

    # Add remaining elements
    result.extend(left[i:])
    result.extend(right[j:])

    return result

# Example usage
data = [38, 27, 43, 3, 9, 82, 10]
print(merge_sort(data))  # Output: [3, 9, 10, 27, 38, 43, 82]

Why Merge Sort Matters

Merge sort's consistent O(n log n) performance and stability (equal elements maintain relative order) make it ideal for:

Sorting linked lists (no random access needed)
External sorting (sorting data larger than RAM (Random-Access Memory))
Any scenario where worst-case guarantee matters

Quicksort

Quicksort is the practical workhorse of sorting algorithms. While its worst-case is O(n²), its average O(n log n) performance and in-place sorting make it the default choice in many languages.

How Quicksort Works

Quicksort uses divide-and-conquer by selecting a pivot element and partitioning the array around it. Elements smaller than the pivot go left; larger go right. Recursively sort each partition.

Quicksort Complexity

Time: O(n log n) average; O(n²) worst case (poor pivot selection)
Auxiliary space for the in-place partition version: O(log n) average (call stack); O(n) worst case.
Auxiliary space for the list-building version below: O(n) for balanced partitions; O(n²) in the worst case because recursive calls retain their lists.

Quicksort in Python

def quicksort(arr):
    """
    Return a sorted list using quicksort (allocates new lists).

    Args:
        arr: list to sort

    Returns:
        Sorted list
    """
    if len(arr) <= 1:
        return arr

    # Choose pivot (using middle element for better average case)
    pivot = arr[len(arr) // 2]

    # Partition into three groups
    left = [x for x in arr if x < pivot]
    middle = [x for x in arr if x == pivot]
    right = [x for x in arr if x > pivot]

    # Recursively sort and combine
    return quicksort(left) + middle + quicksort(right)

def quicksort_inplace(arr, low=0, high=None):
    """
    In-place variant of quicksort (more memory efficient).

    Args:
        arr: list to sort
        low, high: subarray boundaries
    """
    if high is None:
        high = len(arr) - 1

    if low < high:
        # Partition and get pivot position
        pivot_index = partition(arr, low, high)
        # Recursively sort left and right
        quicksort_inplace(arr, low, pivot_index - 1)
        quicksort_inplace(arr, pivot_index + 1, high)

    return arr

def partition(arr, low, high):
    """Partition array around pivot."""
    pivot = arr[high]
    i = low - 1

    for j in range(low, high):
        if arr[j] < pivot:
            i += 1
            arr[i], arr[j] = arr[j], arr[i]

    arr[i + 1], arr[high] = arr[high], arr[i + 1]
    return i + 1

# Example usage
data = [64, 34, 25, 12, 22, 11, 90]
print(quicksort(data))  # Output: [11, 12, 22, 25, 34, 64, 90]

Why Quicksort Matters

Quicksort's in-place partitioning and cache behavior make it useful in production systems. Introsort combines quicksort with a worst-case fallback. Python's Timsort belongs to the merge-sort family, not the quicksort family; CPython's implementation uses a Powersort merge policy. See the CPython sorting design notes.

Breadth-First Search (BFS)

Breadth-First Search is a graph traversal algorithm that explores all vertices at distance k before exploring vertices at distance k+1, where distance counts edges. It finds shortest paths in unweighted graphs (or graphs with equal edge weights), connected components, and level-order traversals.

Given the depth of BFS and graph algorithms in general, we've covered this thoroughly in our dedicated graphs and graph algorithms article, including implementation, applications, and comparison with Depth-First Search.

BFS is a foundation for more advanced techniques like Dijkstra's algorithm (shortest path), level-order tree traversal, and multi-source shortest path problems.

When to Use Each Algorithm

Algorithm	Best For	Key Advantage
Karatsuba	Very large integer multiplication	Faster than naive O(n²) approach
Binary Search	Searching sorted data	Logarithmic O(log n) complexity
Merge Sort	Need consistent O(n log n); stability matters	Guaranteed performance, stable
Quicksort	General-purpose sorting	Fast average case, in-place
BFS	Shortest paths, level-order exploration	Explores by distance layers

References

Introduction to Algorithms (CLRS) — Cormen, Leiserson, Rivest, Stein. MIT Press. The definitive algorithms textbook covering all these algorithms in depth.
Khan Academy: Algorithms Course — https://www.khanacademy.org/computing/computer-science/algorithms. Free, interactive explanations of classic algorithms with visualizations.
Python Documentation: Time Complexity — https://wiki.python.org/moin/TimeComplexity. Official Python performance characteristics for built-in data structures and operations.
Visualgo: Algorithm Visualizer — https://visualgo.net/. Interactive visualizations of sorting, searching, and graph algorithms in action.
Big-O Cheat Sheet — https://www.bigocheatsheet.com/. Quick reference for time and space complexity of common algorithms.

Artificial Vision and Computer Vision: From Pixels to Understanding

Sun, 22 Mar 2026 00:00:00 GMT

Introduction

Computer vision, often called artificial vision, is the field of artificial intelligence that enables computers to interpret and understand the visual world. It processes digital images and videos to extract meaningful information, make decisions, and take actions based on visual input. From medical diagnoses to autonomous vehicles navigating highways, computer vision powers many of today's most transformative technologies.

At its core, computer vision asks a fundamental question: how can we teach machines to "see" and understand images the way humans do? While human vision is intuitive, the computational challenges are immense. A digital image is just a matrix of numbers—pixel values—and extracting semantic meaning from these numbers requires sophisticated mathematical and computational techniques.

This article explores the foundational concepts of computer vision and image processing, starting with basic spatial domain operations and progressing to more advanced frequency domain techniques. We'll examine practical applications and provide working code examples using OpenCV, a widely used computer vision library with Python bindings. The examples require OpenCV, NumPy and Matplotlib, input images at the stated paths, and a desktop GUI (Graphical User Interface) for cv2.imshow. They do not run unchanged in the browser runner. Later fragments reuse variables from earlier examples.

Understanding Visual Computing: Three Distinct Disciplines

Visual computing encompasses three closely related but distinct domains, each with different goals and methodologies.

Image Processing: Enhancing and Analyzing Images

Image processing is the transformation of an image to enhance it or extract useful information. It takes one image as input and produces another image (or data about that image) as output. Image processing focuses on improving image quality, extracting features, or preparing images for further analysis.

Common image processing tasks include:

Noise reduction and image denoising
Contrast enhancement
Feature extraction and edge detection
Image compression and restoration
Color space conversions

Image processing is domain-agnostic—the same techniques work across medical images, photographs, satellite imagery, and microscopy.

Computer Graphics: From 3D Models to 2D Display

Computer graphics creates or manipulates visual content, including 2D drawings and 3D rendering. In 3D rendering, models and scenes produce 2D images for display. Graphics rendering involves lighting calculations, texture mapping, perspective projection, and visual effects.

While computer vision asks "what is in this image?", graphics asks "how do I create this image?" Applications include:

3D scene rendering
Video game engines
Scientific visualization
Virtual and augmented reality

Computer Vision: Understanding and Interpreting Images

Computer vision goes beyond image processing to achieve understanding. It analyzes images to recognize objects, extract semantic meaning, track motion, reconstruct 3D scenes, and make intelligent decisions. Vision tasks include:

Object detection and classification
Facial recognition
Scene understanding
3D reconstruction
Motion estimation

Image Processing in the Spatial Domain

Spatial domain processing operates directly on pixel values and their spatial relationships. Unlike frequency domain methods (which we'll discuss later), spatial processing doesn't transform images into other representations.

Geometric Transformations

Geometric transformations modify the spatial arrangement of pixels. These operations change position, size, or orientation without altering pixel intensities (in the ideal case).

Rotation

Rotation pivots an image around a center point by a specified angle. When rotating, we face a fundamental challenge: pixel coordinates in the rotated image often fall between integer positions of the original image, requiring interpolation.

import cv2
import numpy as np
import matplotlib.pyplot as plt

# Read image
image = cv2.imread('photo.jpg')
height, width = image.shape[:2]
center = (width // 2, height // 2)

# Rotate 45 degrees clockwise (negative angle)
rotation_matrix = cv2.getRotationMatrix2D(center, -45, 1.0)
rotated = cv2.warpAffine(image, rotation_matrix, (width, height))

cv2.imshow('Rotated Image', rotated)
cv2.waitKey(0)

The rotation matrix is a 2×3 affine transformation matrix that OpenCV applies using warpAffine. The third parameter (1.0) is a scale factor for zooming during rotation.

Scaling (Zoom)

Scaling resizes an image by a factor. Upscaling (enlarging) requires interpolation. Downscaling also resamples the image; area interpolation is often useful for reducing aliasing, but not every resize method averages neighborhoods.

# Scale to 50% of original size
scaled_down = cv2.resize(image, (width // 2, height // 2),
                         interpolation=cv2.INTER_AREA)

# Scale to 200% of original size
scaled_up = cv2.resize(image, (width * 2, height * 2),
                       interpolation=cv2.INTER_CUBIC)

Different interpolation methods offer speed-quality tradeoffs:

INTER_NEAREST: Preserves discrete labels; may look blocky on photos.
INTER_LINEAR: Bilinear interpolation, a common general-purpose option.
INTER_AREA: Area-based resampling, often useful for shrinking.
INTER_CUBIC and INTER_LANCZOS4: Use larger neighborhoods and may improve some enlargements, but can introduce ringing. There is no universal best method. See OpenCV geometric transforms.

Translation

Translation shifts an image horizontally and vertically using a 2×3 translation matrix.

# Translate 100 pixels right and 50 pixels down
translation_matrix = np.float32([[1, 0, 100], [0, 1, 50]])
translated = cv2.warpAffine(image, translation_matrix, (width, height))

Histogram Processing and Equalization

A histogram represents the distribution of pixel intensity values. For grayscale 8-bit images, it shows how many pixels have each intensity from 0 (black) to 255 (white).

Histogram equalization redistributes pixel intensities to use the full range more uniformly, improving contrast in low-contrast images.

import cv2
import numpy as np
import matplotlib.pyplot as plt

# Read in grayscale
gray = cv2.imread('photo.jpg', cv2.IMREAD_GRAYSCALE)

# Apply histogram equalization
equalized = cv2.equalizeHist(gray)

# For color images, convert to HSV and equalize the V channel
color_image = cv2.imread('photo.jpg')
hsv = cv2.cvtColor(color_image, cv2.COLOR_BGR2HSV)
hsv[:, :, 2] = cv2.equalizeHist(hsv[:, :, 2])
equalized_color = cv2.cvtColor(hsv, cv2.COLOR_HSV2BGR)

# Compare histograms visually
fig, axes = plt.subplots(1, 2, figsize=(12, 4))
axes[0].hist(gray.ravel(), 256, [0, 256])
axes[0].set_title('Original Histogram')
axes[1].hist(equalized.ravel(), 256, [0, 256])
axes[1].set_title('Equalized Histogram')
plt.show()

Histogram equalization works by computing a cumulative distribution function (CDF) and using it to map old intensity values to new ones, spreading them across the full range. This technique is particularly valuable for medical imaging, where subtle intensity differences may contain diagnostic information.

Arithmetic Operations on Images

Arithmetic operations combine two or more images element-wise or apply mathematical functions to pixel values.

Addition

Adding images can blend or composite them together:

image1 = cv2.imread('photo1.jpg').astype(np.float32)
image2 = cv2.imread('photo2.jpg').astype(np.float32)

# Weighted average: 70% image1, 30% image2
blended = cv2.addWeighted(image1, 0.7, image2, 0.3, 0)

# Convert back to uint8 for display
blended = np.clip(blended, 0, 255).astype(np.uint8)
cv2.imshow('Blended', blended)

Image addition is useful for averaging multiple noisy measurements of the same scene to reduce noise.

Subtraction

Subtracting images reveals differences:

# Detect motion between frames
frame1 = cv2.imread('frame1.jpg').astype(np.float32)
frame2 = cv2.imread('frame2.jpg').astype(np.float32)

difference = cv2.absdiff(frame1, frame2)
# Keep intensity units: frame inputs are 8-bit images converted to float32.
difference = cv2.cvtColor(difference, cv2.COLOR_BGR2GRAY)
difference = np.clip(difference, 0, 255).astype(np.uint8)

# Use threshold to identify regions of significant change
_, motion_mask = cv2.threshold(difference, 30, 255, cv2.THRESH_BINARY)
cv2.imshow('Motion', motion_mask)

Difference images highlight changes and are fundamental to motion detection and change analysis.

Multiplication

Pixel-wise multiplication can apply gain or create masks:

# Create a mask (values 0-1)
mask = (image > 100).astype(np.float32)

# Apply mask to isolate bright regions
masked_image = image.astype(np.float32) * mask
masked_image = np.clip(masked_image, 0, 255).astype(np.uint8)

Spatial Filtering and Convolution

Spatial filters process each pixel using information from its neighborhood. Linear filters can be implemented using convolution or correlation. Nonlinear filters, such as the median filter, are not convolutions.

Convolution Concept

Convolution slides a small kernel (filter matrix) over the image, computing a weighted sum at each position. For a 2D image, the discrete convolution is:

Convolution: Output[i,j] = Σu Σv Kernel[u,v] × Image[i-u, j-v]
Correlation: Output[i,j] = Σu Σv Kernel[u,v] × Image[i+u, j+v]

The indices here are offsets from the kernel center. Convolution flips the kernel; correlation does not. cv2.filter2D and the explorable below use correlation. A symmetric blur kernel gives the same result either way; an asymmetric derivative kernel can change sign. See the OpenCV filtering reference.

The kernel defines what features the filter detects or what effect it applies.

Figure: A Sobel x kernel on a bright square. Pick any output pixel to see the 3×3 window it reads and the weighted sum it computes. The response is strongly positive on the square's left edge, negative on its right edge, and zero where the window sees no horizontal change. (interactive version in the post)

Common Spatial Filters

Blur (Averaging Filter)

A simple averaging kernel reduces noise but also softens edges:

# Gaussian blur: weighted average with Gaussian kernel
blurred = cv2.GaussianBlur(image, (5, 5), 1.0)

# Simple box blur: equal weights
box_blurred = cv2.boxFilter(image, -1, (5, 5))

# Median filter: excellent for salt-and-pepper noise
median_filtered = cv2.medianBlur(image, 5)

Edge Detection

Edges are detected by computing image gradients. The Sobel operator approximates derivatives:

gray = cv2.cvtColor(image, cv2.COLOR_BGR2GRAY)

# Compute horizontal and vertical gradients
sobel_x = cv2.Sobel(gray, cv2.CV_64F, 1, 0, ksize=3)
sobel_y = cv2.Sobel(gray, cv2.CV_64F, 0, 1, ksize=3)

# Magnitude of gradient
edge_magnitude = np.sqrt(sobel_x**2 + sobel_y**2)
maximum = edge_magnitude.max()
if maximum > 0:
    edge_magnitude = edge_magnitude / maximum * 255
edge_magnitude = edge_magnitude.astype(np.uint8)

# Canny edge detector (more sophisticated)
smoothed = cv2.GaussianBlur(gray, (5, 5), 1.4)
edges = cv2.Canny(smoothed, 100, 200)

cv2.imshow('Edges', edges)

The conceptual Canny pipeline smooths the image, computes gradients, performs non-maximum suppression, and uses hysteresis thresholding—it's a complete edge detection pipeline. The example smooths explicitly before calling cv2.Canny.

Sharpening

Sharpening enhances edges by subtracting a blurred version from the original:

# Unsharp mask: sharpening via subtraction
blurred = cv2.GaussianBlur(gray, (0, 0), 2.0)
sharpened = cv2.addWeighted(gray, 1.5, blurred, -0.5, 0)
sharpened = np.clip(sharpened, 0, 255).astype(np.uint8)

Image Processing in the Frequency Domain

While spatial domain processing is intuitive and direct, frequency domain analysis provides powerful insights by decomposing images into sine and cosine waves of different frequencies.

Introduction to the Fourier Transform

The Fourier transform converts an image from the spatial domain (pixel intensity as a function of position) to the frequency domain (amplitude and phase as a function of frequency). This fundamental tool reveals which patterns and frequencies dominate an image.

The 2D Discrete Fourier Transform (DFT) is:

F[u,v] = Σ(x=0..M-1) Σ(y=0..N-1) f[x,y] × exp(-2πi(ux/M + vy/N))

where f[x,y] is the pixel value and F[u,v] is the frequency component.

Why Use the Frequency Domain?

Frequency domain processing offers advantages:

Efficient filtering: Some operations are faster in the frequency domain
Insight: Visualizing the frequency spectrum reveals image properties
Compression: Many compression algorithms work in the frequency domain (e.g., JPEG (Joint Photographic Experts Group, the committee that created the format) uses DCT (Discrete Cosine Transform))
Analysis: Periodic patterns and noise characteristics become obvious

import numpy as np
import matplotlib.pyplot as plt

# Compute FFT
image_gray = cv2.cvtColor(image, cv2.COLOR_BGR2GRAY).astype(np.float32)
fft = np.fft.fft2(image_gray)
fft_shifted = np.fft.fftshift(fft)  # Move zero frequency to center

# Magnitude spectrum (logarithmic scale for visualization)
magnitude = 20 * np.log1p(np.abs(fft_shifted))

# Phase spectrum
phase = np.angle(fft_shifted)

# Display
plt.figure(figsize=(12, 4))
plt.subplot(1, 2, 1)
plt.imshow(magnitude, cmap='gray')
plt.title('Magnitude Spectrum')
plt.colorbar()

plt.subplot(1, 2, 2)
plt.imshow(phase, cmap='gray')
plt.title('Phase Spectrum')
plt.colorbar()
plt.show()

The magnitude spectrum shows which frequencies are strong; the phase carries information about where features are located. Phase-only reconstructions can preserve recognizable structure for some images, but both magnitude and phase are needed for exact reconstruction; this is not a universal ranking of importance.

Practical Example: Building a Complete Image Processing Pipeline

Let's combine multiple techniques in a realistic scenario: enhancing a low-contrast grayscale photograph. This is an image-processing demonstration, not a clinically validated medical-imaging workflow.

import cv2
import numpy as np
import matplotlib.pyplot as plt

# Read image
image = cv2.imread('photo.jpg', cv2.IMREAD_GRAYSCALE)
if image is None:
    raise FileNotFoundError('photo.jpg')

# Step 1: Denoise using bilateral filter
# (preserves edges while smoothing)
denoised = cv2.bilateralFilter(image, 9, 75, 75)

# Step 2: Enhance contrast using CLAHE
# (Contrast Limited Adaptive Histogram Equalization)
clahe = cv2.createCLAHE(clipLimit=2.0, tileGridSize=(8, 8))
enhanced = clahe.apply(denoised)

# Step 3: Edge enhancement using high-pass filter
blurred = cv2.GaussianBlur(enhanced, (0, 0), 2.0)
high_pass = enhanced.astype(np.float32) - blurred.astype(np.float32)
sharpened = np.clip(enhanced.astype(np.float32) + 0.5 * high_pass, 0, 255)

# Step 4: Normalize intensity
normalized = cv2.normalize(sharpened, None, 0, 255, cv2.NORM_MINMAX).astype(np.uint8)

# Display results
fig, axes = plt.subplots(2, 3, figsize=(15, 10))
axes[0, 0].imshow(image, cmap='gray')
axes[0, 0].set_title('Original')
axes[0, 1].imshow(denoised, cmap='gray')
axes[0, 1].set_title('Denoised')
axes[0, 2].imshow(enhanced, cmap='gray')
axes[0, 2].set_title('Enhanced')
axes[1, 0].imshow(high_pass, cmap='gray')
axes[1, 0].set_title('High-Pass Filter')
axes[1, 1].imshow(sharpened, cmap='gray')
axes[1, 1].set_title('Sharpened')
axes[1, 2].imshow(normalized, cmap='gray')
axes[1, 2].set_title('Normalized')

for ax in axes.flat:
    ax.axis('off')
plt.tight_layout()
plt.show()

This pipeline demonstrates how combining multiple techniques—denoising, contrast enhancement, sharpening, and normalization—changes an image. Whether it helps depends on the task; it may also amplify noise or distort useful information.

Common Applications of Computer Vision

Computer vision technology powers countless real-world applications:

Medical Imaging

Computer vision assists in diagnosis and treatment planning. Algorithms detect tumors, segment organs, measure anatomical structures, and register images across time or modalities (CT (Computed Tomography), MRI (Magnetic Resonance Imaging), ultrasound). Performance depends on the specific task, data and clinical validation; success on a benchmark does not establish safe use in a clinical workflow.

Autonomous Driving

Self-driving vehicles rely on multiple cameras and sophisticated vision algorithms. Tasks include lane detection, vehicle and pedestrian detection, traffic sign recognition, and 3D scene understanding from monocular or stereo cameras. Real-time performance is critical—decisions must be made in milliseconds.

Quality Control and Inspection

Manufacturing facilities use vision systems for automated quality inspection. Cameras detect defects, measure dimensions, verify component assembly, and identify damaged products at production speeds—faster and more consistently than human inspectors.

Robotics and Manipulation

Robots use vision for navigation, obstacle avoidance, object grasping, and hand-eye coordination. Industrial robots identify parts, calculate grasp poses, and verify task completion using sophisticated vision systems.

Surveillance and Security

Security systems analyze video feeds for intrusion detection, crowd monitoring, anomaly detection, and forensic investigation. Modern systems use deep learning for person tracking, behavior analysis, and event detection.

Face Recognition and Biometrics

Face detection and recognition unlock phones, control access to buildings, identify individuals in photos, and power law enforcement applications. Similar techniques work for iris recognition, fingerprint analysis, and gait recognition.

Remote Sensing and GIS

Satellite and aerial imagery analysis supports agriculture (crop monitoring, yield prediction), urban planning, environmental monitoring, disaster response, and climate change analysis.

Conclusion

Computer vision transforms raw pixels into actionable insights and intelligent decisions. Understanding the foundational techniques—geometric transformations, histogram processing, spatial filtering, and frequency domain analysis—provides the conceptual foundation for more advanced methods like deep learning.

Modern computer vision increasingly relies on deep neural networks that automatically learn visual features from data. However, the classical techniques discussed here remain important: they're computationally efficient, interpretable, and foundational to understanding how modern methods work.

Whether you're processing medical images, building autonomous systems, or developing creative visual applications, the principles and code examples in this article provide a practical starting point for your computer vision journey.

References

OpenCV Documentation: https://docs.opencv.org/ - Comprehensive reference for the OpenCV library with tutorials and API (Application Programming Interface) documentation.
Gonzalez, R. C., & Woods, R. E. (2018). Digital Image Processing (4th ed.). Pearson. - The definitive textbook on digital image processing, covering both spatial and frequency domain techniques with rigorous mathematical foundations.
Stanford CS231n: Convolutional Neural Networks for Visual Recognition: https://cs231n.stanford.edu/ - Excellent course covering image classification, feature extraction, and the connection between classical image processing and modern deep learning.
Bradski, G., & Kaehler, A. (2008). Learning OpenCV: Computer Vision with the OpenCV Library. O'Reilly Media. - Practical guide to OpenCV with real-world examples and applications.
Forsyth, D. A., & Ponce, J. (2003). Computer Vision: A Modern Approach. Pearson. - Comprehensive textbook covering geometry, features, recognition, and 3D vision.

Dynamic Programming

Sun, 22 Mar 2026 00:00:00 GMT

Dynamic programming is a powerful technique used to solve complex problems by breaking them down into simpler sub-problems and reusing partial results.

It is built on two ideas:

Optimal substructure: the best answer for a problem can be built from the best answers of smaller sub-problems.
Overlapping sub-problems: the same smaller problems appear repeatedly.

These properties can make reuse effective, but polynomial running time also requires a polynomial number of states and work per state. Some dynamic programs remain exponential or pseudopolynomial. Counting problems such as Fibonacci use the same approach without an optimization objective.

The Fibonacci examples assume a nonnegative integer n. Their O(n) time and O(1) optimized space count arithmetic operations and stored integers; Python's arbitrary-precision integers grow with n, so bit costs are larger.

Why Dynamic Programming Matters

Many algorithms re-compute the same values over and over. Dynamic programming stores those intermediate results so each one is computed only once.

This often gives large performance gains:

Time: from $O(2^n)$ to $O(n)$ or $O(n^2)$ in many classic cases.
Space: additional memory is used to save computed states.

The key trade-off is clear: we spend memory to save time.

A Useful Mental Model

To solve a dynamic programming problem, define the following pieces first:

State: what sub-problem are you solving?
Transition: how does one state depend on previous states?
Base cases: what states are already known?
Answer: which state represents the final solution?

If you can clearly write these four parts, implementation becomes much easier.

Top-Down Approach (Memoization)

Top-down dynamic programming starts with the final question and recursively solves smaller questions, caching answers as they are discovered.

Fibonacci with Memoization

from functools import lru_cache


@lru_cache(maxsize=None)
def fibonacci(n: int) -> int:
  if n <= 1:
    return n
  return fibonacci(n - 1) + fibonacci(n - 2)


print(fibonacci(10))  # 55

Using a cache avoids repeated calls like fibonacci(5) being computed many times.

Figure: The naive fibonacci(5) makes 15 calls. With @lru_cache, each value from f(0) to f(5) is computed once, the 3 repeats become cache hits, and the 6 calls below them never happen. (interactive version in the post)

Bottom-Up Approach (Tabulation)

Bottom-up dynamic programming starts from base cases and iteratively builds toward the final answer.

Fibonacci with Tabulation

def fibonacci(n: int) -> int:
  if n <= 1:
    return n

  dp = [0] * (n + 1)
  dp[1] = 1

  for i in range(2, n + 1):
    dp[i] = dp[i - 1] + dp[i - 2]

  return dp[n]


print(fibonacci(10))  # 55

This version avoids recursion overhead and is often preferred in performance sensitive paths.

Space Optimization Example

Some dynamic programming transitions only depend on a few previous states. In that case, you can store only those values.

def fibonacci(n: int) -> int:
  if n <= 1:
    return n

  prev2, prev1 = 0, 1
  for _ in range(2, n + 1):
    prev2, prev1 = prev1, prev1 + prev2

  return prev1

This reduces the number of stored Fibonacci values from $O(n)$ to $O(1)$.

Another Classic Problem: Minimum Coin Change

Problem: given coin values and a target amount, find the minimum number of coins needed to make that amount, with unlimited copies of each coin. Assume positive integer coin values and a nonnegative integer amount. The time cost is O(amount × number of denominations), pseudopolynomial in the amount's bit length.

State definition:

dp[a] = minimum number of coins to make amount a.

Transition:

dp[a] = min(dp[a], dp[a - coin] + 1) for each valid coin.

def min_coins(coins: list[int], amount: int) -> int:
  inf = amount + 1
  dp = [inf] * (amount + 1)
  dp[0] = 0

  for a in range(1, amount + 1):
    for coin in coins:
      if coin <= a:
        dp[a] = min(dp[a], dp[a - coin] + 1)

  return dp[amount] if dp[amount] != inf else -1


print(min_coins([1, 3, 4], 6))  # 2 (3 + 3)

Common Pitfalls

Defining the wrong state.
Forgetting base cases.
Missing transitions.
Using mutable default arguments in recursive functions.
Not checking memory usage when the state space is large.

How to Decide If DP Applies

Ask these questions:

Can the problem be split into smaller versions of itself?
Do those smaller versions repeat?
Can I combine smaller optimal answers into a larger optimal answer?

If the answer is yes to all three, dynamic programming is likely a good fit.

Final Thoughts

Dynamic programming is not one algorithm, but a way of thinking. Once you practice state design and transitions, many hard problems become manageable and systematic to solve.

Economía Básica

Sun, 22 Mar 2026 00:00:00 GMT

Palabras claves: economía, oferta, demanda, competencia perfecta, monopolio, dinero, tiempo, valor

La economía es el conjunto de actividades (transacciones) que se realizan para satisfacer las necesidades de las personas. Estas actividades incluyen la producción, la distribución, el intercambio y el consumo de bienes y servicios. Entender los principios fundamentales de la economía es esencial para tomar decisiones financieras informadas, tanto a nivel personal como profesional.

Transacciones

Una transacción es el canje de bienes y servicios entre dos o más partes. En esencia, toda transacción involucra un intercambio de valor donde ambas partes esperan mejorar su situación.

Las transacciones modernas se realizan utilizando dinero y/o crédito como medio de intercambio. Estos recursos se intercambian por activos financieros, mercancías, servicios y otros bienes de valor. Los participantes principales en las transacciones económicas son las empresas, los gobiernos y los bancos, aunque también incluye a individuos y consumidores.

El gasto es un flujo durante un período, mientras que el dinero y el crédito disponible se miden como saldos en un momento. Por eso no existe la identidad «dinero + crédito = gasto total»: una misma unidad monetaria puede gastarse varias veces y una línea de crédito puede no utilizarse. El crédito puede financiar gasto, pero no se convierte automáticamente en él.

Mercado

Un mercado es el lugar (físico o virtual) donde se realizan transacciones de compra y venta del mismo producto o servicio. Los mercados son mecanismos eficientes para que compradores y vendedores se encuentren y acuerden precios.

Existen mercados para prácticamente todo: desde bienes de consumo hasta valores financieros, servicios profesionales y activos reales. La característica principal de un mercado es que permite la formación de precios a través de la interacción entre oferta y demanda, creando un equilibrio entre lo que los productores quieren vender y lo que los consumidores quieren comprar.

Los mercados pueden ser locales, nacionales o internacionales. También pueden ser más o menos regulados dependiendo del país y el tipo de bien o servicio que se comercialice. La información disponible en el mercado, la transparencia de precios y la facilidad para participar influyen en la eficiencia del mercado.

Crédito

El crédito es un acuerdo para recibir recursos hoy y pagarlos después. Puede financiar una transacción, pero no es sinónimo de dinero. Al conceder un préstamo, un banco comercial normalmente crea un depósito y un activo crediticio; su capacidad está limitada por regulación, riesgo, rentabilidad y demanda. Véase Money creation in the modern economy, del Banco de Inglaterra.

Los participantes en operaciones de crédito son los prestadores (o acreedores) y los prestatarios (o deudores). El prestador proporciona recursos con la expectativa de ser reembolsado en el futuro, generalmente con intereses. El prestatario recibe recursos hoy a cambio de la obligación de devolver una cantidad mayor en el futuro.

El crédito actúa como un poderoso motor de crecimiento económico a corto plazo. Permite a empresas e individuos invertir en proyectos, comprar bienes y expandir operaciones sin esperar a acumular ahorros. Sin embargo, este crecimiento acelerado viene con riesgos: si el crecimiento futuro no materializa o el prestatario no puede devolver el crédito, se produce una crisis de endeudamiento.

Tasa de Interés

La tasa de interés es el precio del dinero. Puede expresar tanto la rentabilidad de un activo financiero como el costo de un crédito. Es uno de los instrumentos más importantes que tiene un banco central para regular la economía.

Para que una economía se mantenga estable, la inflación debe mantenerse dentro de un rango establecido por el banco central. Esto es especialmente importante para preservar el poder adquisitivo de la moneda. El Índice de Precios al Consumidor (IPC) es el indicador clave que los bancos centrales monitorean. Cuando la inflación sube demasiado, el banco central aumenta la tasa de interés para reducir la cantidad de dinero en circulación y disminuir la demanda. Cuando la inflación es demasiado baja, el banco central disminuye la tasa de interés para estimular el gasto y la inversión.

En un préstamo simple sin pagos intermedios ni comisiones, el importe total a devolver es principal + intereses. Eso no define el crédito en general. El banco central de cada país establece la tasa de interés de referencia, que sirve como base para todas las otras tasas en la economía. Esta tasa es una herramienta fundamental para conservar el poder adquisitivo de la moneda y controlar tanto la inflación como el crecimiento económico.

De forma simplificada, y con rezagos y efectos que dependen del contexto:

Tasa de interés baja → más crédito disponible → más gasto → más producción (activa la economía)
Tasas de interés altas → menos crédito disponible → menos gasto → menos producción (frena la economía)

Un concepto importante es que el crédito se convierte en deuda. Para el prestador es un activo (dinero que le deben), mientras que para el prestatario es un pasivo (dinero que debe pagar).

Ciclo Económico

Los ciclos económicos son fluctuaciones de la actividad agregada. Una clasificación habitual distingue cuatro etapas o puntos de giro:

Expansión: aumenta la actividad económica.
Pico: nivel máximo antes de la contracción; no es la máxima tasa de crecimiento.
Contracción o recesión: descenso significativo de la actividad; crecer más despacio no equivale por sí solo a una recesión.
Valle: mínimo desde el que empieza la siguiente expansión.

La recuperación es parte de la expansión. Una depresión es una contracción especialmente profunda y prolongada, no una etapa obligatoria de cada ciclo. La duración y las causas varían: pueden intervenir crédito, demanda, política monetaria y fiscal, tecnología o choques externos. Véanse las definiciones del ciclo de la NBER (National Bureau of Economic Research (Oficina Nacional de Investigación Económica de EE. UU.)).

Crecimiento Económico

El crecimiento de la producción depende, entre otros factores, de la cantidad de trabajo, la acumulación de capital y la productividad. El crédito puede financiar inversión que amplíe esa capacidad o gasto corriente; sus efectos dependen de qué se financie y de la capacidad de pago.

Productividad y deuda no son fuerzas excluyentes. Una empresa puede endeudarse para mejorar su productividad, y tanto la innovación como el crédito pueden acompañar fluctuaciones económicas. La productividad es especialmente importante para el crecimiento sostenido del ingreso por persona.

Productividad

La productividad mide cuánto producimos y, en última instancia, determina cuánto podemos consumir de forma sostenible. Es la base real del bienestar económico: el verdadero soporte del crecimiento económico a largo plazo.

Existen diferentes tipos de productividad: productividad del trabajo (cuánto produce un trabajador por hora), productividad del capital (cuánto ingreso genera una unidad de capital), y productividad total de factores (que mide la eficiencia con la que se combinan trabajo, capital y otros recursos).

Aumentar la productividad requiere inversión en educación, tecnología, infraestructura y mejora de procesos. Los países con mayor productividad tienden a tener mayores ingresos per cápita, mejores salarios y mayor calidad de vida. La inversión en productividad es la vía sostenible hacia el bienestar económico.

Ciclo de Deuda a Corto Plazo

El ciclo de deuda a corto plazo es un mecanismo fundamental que impulsa la economía en períodos de expansión:

Crédito → Deuda → Gasto → Producción → Ingreso → Más Crédito

No hay una duración fija ni una periodicidad garantizada para este mecanismo. Cuando los bancos otorgan más crédito, las personas y empresas gastan más, lo que impulsa la producción, aumenta el empleo y genera más ingreso. Con más ingreso, la capacidad de tomar más crédito aumenta, alimentando el ciclo. Sin embargo, este ciclo no es infinito: eventualmente, la deuda acumulada se vuelve demasiado pesada de servir, y el ciclo puede invertirse y contribuir a una recesión; no toda expansión crediticia termina necesariamente así.

Ciclo de Deuda a Largo Plazo

Mientras que el ciclo corto dura algunos años, el ciclo de deuda a largo plazo puede extenderse durante décadas. Este ciclo refleja la acumulación sostenida de deuda en la economía, con períodos largos de crecimiento seguidos de ajustes severos.

El ciclo largo se construye sobre múltiples ciclos cortos. Cada ciclo corto puede dejar un residuo de deuda que no se liquida completamente, acumulándose gradualmente. Cuando esta deuda acumulada se vuelve insostenible en relación al ingreso, ocurre una desaceleración severa o una "gran recesión" que puede durar varios años.

El fin de un ciclo de deuda largo requiere generalmente de intervenciones significativas: reducción de deuda (quitas), reestructuración de deudas, aumento de productividad para que la economía crezca más rápido que la deuda, o una combinación de estos factores. Comprender estos ciclos largos es esencial para interpretar riesgos, pero no permite fechar de forma fiable la próxima crisis.

Costo de Oportunidad

El costo de oportunidad es el valor de la mejor alternativa que no elegimos. Cuando decidimos hacer algo, renunciamos automáticamente a todas las otras cosas que podríamos haber hecho en ese mismo tiempo o con esos mismos recursos.

Por ejemplo, si inviertes tu dinero en acciones de una empresa, el costo de oportunidad es el retorno que hubieras obtenido si lo hubieras invertido en bonos, bienes raíces, o dejado en una cuenta de ahorros. Este concepto es fundamental para tomar buenas decisiones económicas porque nos obliga a comparar realmente todas las opciones disponibles.

Entender el costo de oportunidad nos ayuda a ser más racionales en nuestras decisiones. No se trata solo de si algo es bueno en términos absolutos, sino si es mejor que las alternativas disponibles.

Distribución del Ingreso

La distribución del ingreso es la forma en que se reparte el ingreso entre los miembros de una sociedad. Es un tema fundamental en economía porque determina la desigualdad económica, la accesibilidad a oportunidades y la estabilidad social.

Una distribución muy desigual del ingreso puede indicar una economía donde las oportunidades están concentradas en pocas manos, lo que limita la movilidad social y el crecimiento económico inclusivo. Por el contrario, una distribución más equitativa tiende a promover mayor acceso a educación, salud y oportunidades de negocio para una proporción mayor de la población.

La distribución del ingreso se mide frecuentemente usando el Coeficiente de Gini, que varía de 0 (igualdad perfecta) a 1 (desigualdad máxima). Los gobiernos pueden influir en la distribución del ingreso a través de impuestos progresivos, programas de transferencias sociales, inversión en educación pública y regulación del mercado laboral. Es un tema de gran importancia política, social y económica.

Escasez

La escasez es la falta de recursos suficientes para satisfacer todas las necesidades y deseos de una población. Es un problema económico fundamental que afecta a todos los países, aunque en diferentes grados.

La escasez es lo que hace que la economía sea necesaria: si tuviéramos recursos ilimitados, no habría que tomar decisiones sobre asignación. Todos los bienes económicos son escasos en el sentido de que hay una cantidad limitada disponible. Esta realidad obliga a las sociedades a responder tres preguntas fundamentales: ¿Qué producir?, ¿Cómo producir?, y ¿Para quién producir?

La escasez es relativa, no absoluta. Una sociedad puede ser escasa en algunos recursos pero abundante en otros. Además, la tecnología y la innovación pueden reducir la escasez de ciertos bienes (por ejemplo, la energía solar reduce la escasez de energía).

Oferta y Demanda

La oferta y la demanda son dos conceptos fundamentales que determinan los precios y las cantidades en los mercados. La oferta representa la cantidad de bienes o servicios que los productores están dispuestos a ofrecer en el mercado a diferentes precios. La demanda, por otro lado, representa la cantidad de bienes o servicios que los consumidores desean comprar a diferentes precios.

Manteniendo constantes los demás factores, al subir el precio suele disminuir la cantidad demandada y aumentar la cantidad ofrecida. Son movimientos sobre las curvas. Cambios en ingresos, costos o preferencias desplazan las curvas de demanda u oferta. El precio de equilibrio es aquel en el que la cantidad ofrecida iguala la cantidad demandada.

Cuando la oferta es menor que la demanda, hay escasez y los precios tienden a subir. Cuando la oferta es mayor que la demanda, hay exceso de inventario y los precios tienden a bajar. Este es un modelo de ajuste, no una garantía de equilibrio instantáneo. Información imperfecta, poder de mercado, costos de ajuste, contratos y regulaciones pueden influir en la respuesta.

Figure: Mueve la demanda o la oferta y observa el nuevo equilibrio. Si la demanda sube, suben el precio y la cantidad; si la oferta sube, el precio baja y la cantidad sube. Las líneas punteadas grises son las curvas originales. (interactive version in the post)

Tipos de Mercado

Existen diferentes estructuras de mercado que afectan cómo se forman los precios y cómo se comportan las empresas:

Competencia Perfecta: Muchos vendedores y compradores, productos homogéneos, libre entrada y salida del mercado. En este tipo de mercado, ningún vendedor individual puede influir en el precio. Los precios se determinan por la oferta y demanda del mercado completo. Ejemplos aproximados incluyen mercados agrícolas muy grandes.

Monopolio: Un único vendedor domina el mercado, con barreras significativas a la entrada. El monopolista tiene poder de mercado y puede influir en los precios. Sin embargo, enfrenta la restricción de que a precios más altos, menos personas comprarán. Los monopolios pueden surgir por economías de escala (es más eficiente que una empresa provea un servicio) o por control de recursos clave.

Oligopolio: Pocos vendedores grandes dominan el mercado. Hay interdependencia entre las empresas: cada empresa debe considerar cómo sus decisiones de precio y cantidad afectarán a sus competidores. Ejemplos incluyen la industria de telecomunicaciones, aviación y manufactura automotriz.

Entender la estructura del mercado es importante porque afecta los precios que pagamos, la innovación en la industria, y la rentabilidad de las empresas.

Rentabilidad y Utilidad

La rentabilidad y la utilidad son conceptos relacionados pero diferentes que miden el éxito financiero de una inversión.

La rentabilidad es la relación porcentual entre el beneficio obtenido y la inversión realizada. Es una medida relativa que permite comparar el retorno de diferentes inversiones de diferentes tamaños. Si inviertes 100 pesos y recibes 120 al final, tu rentabilidad es del 20 %. Invertir un millón y recibir 1,2 millones da el mismo porcentaje.

La utilidad, por otro lado, es el montante absoluto de dinero que se obtiene de la venta de un bien, servicio o inversión después de restar los costos. Es una medida en términos de dinero, no de porcentaje. La utilidad contable no equivale necesariamente al efectivo recibido, pero no te permite comparar fácilmente con otras inversiones.

Rentabilidad (%) = (Valor final / Inversión − 1) × 100
Utilidad         = Valor final − Inversión inicial

Para la toma de decisiones, generalmente la rentabilidad es más relevante porque permite comparar oportunidades de diferentes tamaños; también hay que comparar plazos, riesgos, costos y flujos de caja.

Riesgo

El riesgo es la probabilidad de que el resultado real de una inversión sea significativamente diferente al resultado esperado. En otras palabras, es la incertidumbre sobre los retornos futuros.

Existen diferentes tipos de riesgo: el riesgo de mercado (el precio del activo baja), el riesgo de crédito (el deudor no paga), el riesgo operacional (algo sale mal en la empresa), y el riesgo sistemático (toda la economía cae). Los inversores racionales exigen una mayor rentabilidad para asumir mayor riesgo. Esta relación entre riesgo y retorno es fundamental en finanzas.

El riesgo no puede eliminarse completamente en la mayoría de las inversiones, pero puede gestionarse y diversificarse. Una cartera diversificada que incluya diferentes tipos de activos tiende a tener menos riesgo que apostar todo en una única inversión.

Flujo de Caja y Liquidez

El flujo de caja es la cantidad de dinero que entra y sale de una empresa (o persona) durante un período específico de tiempo. Es diferente de las ganancias contables porque se enfoca en dinero real, no en ingresos o gastos reconocidos contablemente.

El flujo de caja puede ser positivo (entra más dinero del que sale) o negativo (sale más dinero del que entra). Una empresa muy rentable en términos contables puede quebrar si tiene un flujo de caja negativo, porque no tendrá efectivo para pagar sus obligaciones inmediatas.

La liquidez es la capacidad de una empresa (o persona) para convertir sus activos en efectivo rápidamente sin perder valor. Un activo muy líquido como el efectivo puede convertirse inmediatamente, mientras que bienes raíces son menos líquidos porque pueden tomar meses vender una propiedad. La liquidez es crítica para sobrevivir a crisis económicas o sorpresas inesperadas.

Deudas y Gastos

Aunque a menudo se confunden, deudas y gastos son conceptos distintos que es importante diferenciar para una buena salud financiera.

Los gastos representan recursos consumidos o costos incurridos durante un período. No todos son voluntarios ni coinciden con un pago inmediato; en contabilidad, una depreciación es un gasto sin salida de efectivo en ese momento. Los gastos pueden ser necesarios (alimento, vivienda) o discrecionales (entretenimiento, lujos).

Las deudas, por otro lado, son obligaciones o compromisos financieros que tenemos con terceros. Representan dinero que nos han prestado y que debemos devolver, generalmente con intereses. Mientras que un gasto es definitivo, una deuda es temporal: eventualmente debe saldarse.

Una deuda se vuelve problemática cuando el flujo de caja no es suficiente para cubrirla. Si nuestro ingreso no crece al ritmo de nuestras deudas y gastos, eventualmente seremos insolventes.

Ahorro e Inversión

El ahorro es la parte del ingreso que no se gasta inmediatamente. Es reservar dinero para el futuro, reduciendo el consumo presente. La inversión, por otro lado, es usar el dinero ahorrado para comprar activos que generen retornos futuros.

En una economía cerrada, el ahorro y la inversión agregados coinciden como identidad contable ex post. En una economía abierta también interviene el financiamiento exterior. Esto no significa que todo préstamo bancario necesite un depósito ahorrado previamente: crear crédito y financiar inversión puede generar ingresos y ahorro después. A nivel personal, el ahorro es la clave para construir riqueza a largo plazo, especialmente mediante la capitalización de intereses compuestos.

La tasa de ahorro (qué porcentaje del ingreso se ahorra) varía significativamente entre países y personas. Países con altas tasas de ahorro tienden a tener más capital disponible para inversión, lo que impulsa el crecimiento. Culturas que valoran el ahorro tienden a acumular más riqueza a largo plazo. Sin embargo, demasiado ahorro puede frenar el crecimiento económico a corto plazo si reduce el consumo excesivamente.

Política Fiscal y Monetaria

La política fiscal y monetaria son las dos herramientas principales que los gobiernos y bancos centrales usan para regular la economía.

Política Monetaria: Es la regulación de la cantidad de dinero en la economía y de las tasas de interés, generalmente bajo control del banco central. Una política monetaria expansiva aumenta la cantidad de dinero (tasas bajas) para estimular la economía durante recesiones. Una política monetaria contractiva reduce la cantidad de dinero (tasas altas) para frenar la inflación durante períodos de sobrecalentamiento económico.

Política Fiscal: Es el uso de gastos e impuestos del gobierno para regular la economía. Una política fiscal expansiva aumenta el gasto público o reduce impuestos para estimular el crecimiento durante recesiones. Una política fiscal contractiva reduce el gasto o aumenta impuestos para enfriar la economía durante inflación. Un déficit o superávit por sí solo no identifica el cambio de orientación fiscal: hay que comparar períodos y separar los efectos del ciclo económico de las decisiones de gasto e impuestos.

Ambas políticas tienen limitaciones y efectos secundarios. La política monetaria puede perder efectividad en depresiones muy severas. La política fiscal requiere tiempo para implementarse y puede tener efectos adversos a largo plazo si causa deuda excesiva. La combinación óptima de ambas políticas es un tema central en macroeconomía.

Reflexiones Finales

El entendimiento de conceptos como oferta y demanda, ciclos de deuda, productividad, y la diferencia entre crecimiento por crédito versus crecimiento por productividad es esencial para navegar el mundo financiero moderno. Ya sea como inversor, emprendedor, trabajador o simplemente ciudadano, estas ideas afectan nuestras vidas diariamente.

Lo importante es reconocer que la economía es cíclica, que la deuda tiene costos, y que la productividad es una fuente central del crecimiento sostenido del ingreso por persona. Las decisiones económicas son también decisiones de oportunidad: elegimos entre alternativas escasas.

Referencias

Comisión para el Mercado Financiero de Chile. "¿Qué es la tasa de interés?". Recuperado de https://www.cmfchile.cl/educa/621/w3-article-27164.html
Investopedia. (2024). "Economics: Definition, Topics, and Schools of Thought". Recuperado de https://www.investopedia.com/terms/e/economics.asp
Principios Fundamentales de la Economía: Costo y Oportunidad. Recuperado de https://platzi.com/cursos/economia-del-comportamiento/los-cuatro-principios-de-la-economia/

Indicadores Económicos: Ocho Números para Leer una Economía

Sun, 22 Mar 2026 00:00:00 GMT

Un país anuncia que su PIB (Producto Interno Bruto (Gross Domestic Product, GDP)) creció 2,35 %. ¿Es una buena noticia? Depende de cuánto subieron los precios, de cuánta gente tiene empleo y de cuánto debe el gobierno. Los indicadores económicos son las estadísticas que responden esas preguntas, y cada uno ilumina solo una parte de la economía.

En este artículo veremos ocho de ellos: qué miden, cómo se calculan con un ejemplo numérico y por qué ninguno cuenta solo la historia completa. La tabla resume lo esencial; el resto del artículo la desarrolla.

Indicador	Qué mide	Ojo con
PIB	Valor de lo producido en un período	Distinguir PIB nominal de PIB real
Inflación (IPC)	Alza general y sostenida de los precios	Desinflación no es deflación
Desempleo	Desempleados sobre la población activa	No refleja la calidad del empleo
Balanza comercial	Exportaciones menos importaciones	Un superávit no equivale a más reservas
Tasa de interés	El costo del dinero	Su efecto depende del plazo y del riesgo
Confianza del consumidor	Optimismo de los hogares	Anticipa el consumo, no lo garantiza
Deuda pública	Lo que el gobierno debe, como % del PIB	Ningún umbral la declara segura o insostenible
Tipo de cambio	El precio de una moneda en otra	Favorece a unos y perjudica a otros

Producto Interno Bruto (PIB): el tamaño de la economía, no su ritmo

El PIB mide el valor de los bienes y servicios finales producidos dentro de un territorio en un período de tiempo determinado, generalmente un año. Es el indicador más amplio de la actividad económica y, por tanto, uno de los más utilizados para evaluar la salud de una economía.

Un PIB alto describe el tamaño de la economía, no su tasa de crecimiento. Para medir cambios en la producción se compara el PIB real, ajustado por precios; el PIB nominal también puede subir por inflación. No hay una tasa universal de crecimiento «sana»: depende de la población, la productividad y el contexto. Véase la explicación del PIB de la BEA (Bureau of Economic Analysis (Oficina de Análisis Económico de EE. UU.)).

Cómo se calcula el PIB

La fórmula más común para calcular el PIB es el enfoque de gasto:

PIB = C + I + G + (X - M)

Donde:

C = Consumo (gasto de los hogares en bienes y servicios)
I = Inversión privada bruta, incluida vivienda y variación de inventarios
G = Compras de bienes y servicios e inversión del gobierno; no incluye transferencias como pensiones, que no pagan producción corriente
X = Exportaciones (bienes y servicios vendidos al exterior)
M = Importaciones (bienes y servicios comprados del exterior)

Ejemplo práctico

Imaginemos un país con los siguientes datos anuales:

Consumo de hogares: $500 mil millones
Inversión empresarial: $150 mil millones
Gasto del gobierno: $200 mil millones
Exportaciones: $100 mil millones
Importaciones: $80 mil millones

El PIB sería: $500 + $150 + $200 + ($100 - $80) = $870 mil millones

Si el año anterior fue $850 mil millones, el crecimiento del PIB es ($870 - $850) / $850 × 100 = 2,35 % anual. Esta es una variación nominal si los valores están a precios corrientes; solo mide crecimiento real si ambos están expresados en precios comparables.

Inflación: el dinero compra menos

La inflación es el aumento generalizado y sostenido de los precios de bienes y servicios en una economía durante un período de tiempo. En otras palabras, la inflación reduce el poder adquisitivo del dinero: cada unidad monetaria compra menos productos que antes.

Los objetivos de inflación varían entre bancos centrales. La inflación elevada o imprevisible dificulta la planificación y reduce el poder de compra si los ingresos no la acompañan. Desinflación significa que la inflación disminuye: si pasa del 8 % al 4 %, los precios siguen subiendo. Deflación significa una caída del nivel general de precios; puede aumentar la carga real de las deudas y, si se espera que continúe, aplazar ciertas compras.

IPC (Índice de Precios al Consumidor)

El IPC es el indicador más utilizado para medir la inflación. Mide la variación de los precios de un conjunto de bienes y servicios que adquiere un consumidor promedio en un período determinado. Este conjunto incluye alimentos, vivienda, transporte, salud, educación, entretenimiento, y más.

En un índice simplificado de canasta fija, el costo de las cantidades del período base se compara a precios actuales y a precios base:

IPC = 100 × Σ(precio_actual × cantidad_base) / Σ(precio_base × cantidad_base)

Si usamos participaciones del gasto base w_i que suman 1, la forma equivalente es IPC = 100 × Σ w_i × (precio_actual_i / precio_base_i). No se deben mezclar esas participaciones con cantidades físicas. Los IPC oficiales tienen métodos de agregación, actualización de ponderaciones y ajuste de calidad más completos; véase la guía del IPC de la BLS (Bureau of Labor Statistics (Oficina de Estadísticas Laborales de EE. UU.)).

Ejemplo práctico del IPC

Supongamos estas participaciones en el gasto del período base:

Bien	Precio base	Participación del gasto base	Precio actual
Leche (litro)	$1,00	10 %	$1,05
Pan (kilo)	$3,00	20 %	$3,15
Gasolina (litro)	$2,00	70 %	$2,10

Cada precio subió un 5 %, así que:

IPC = 100 × (0,10 × 1,05 + 0,20 × 1,05 + 0,70 × 1,05) = 105
Inflación respecto al período base = (105 / 100 − 1) × 100 = 5 %

Factores que producen inflación

La inflación no surge de una única causa, sino de una combinación de factores:

Incremento de la demanda: Cuando hay más demanda de bienes y servicios que oferta disponible, los precios suben. Por ejemplo, durante un auge económico, el consumo aumenta rápidamente y los productores no pueden satisfacer toda la demanda.
Aumento de los costos de producción: Si el costo de las materias primas, el trabajo, o la energía sube, las empresas trasladan estos costos a los precios finales. Un ejemplo es cuando el precio del petróleo sube, incrementando el costo de transporte y producción.
Inflación auto-generada o expectativas inflacionarias: Si los agentes económicos (trabajadores, empresas, consumidores) esperan inflación futura, actúan en consecuencia: los trabajadores piden salarios más altos, las empresas suben precios anticipadamente, y los consumidores compran más rápido. Estas respuestas pueden contribuir a la persistencia de la inflación; no implican que toda expectativa se cumpla automáticamente.
Aumento de la cantidad de dinero en circulación: Si el banco central aumenta mucho la oferta monetaria y la producción no crece igual, hay más dinero persiguiendo los mismos productos. Eso genera presiones inflacionarias y el dinero pierde valor relativo.
Inflación importada: En economías abiertas, si las monedas extranjeras se aprecian (o la moneda local se deprecia), los bienes importados se vuelven más caros, lo que puede transmitirse a la inflación general.

Tasa de Desempleo: cuánta gente busca trabajo y no lo encuentra

La tasa de desempleo mide el porcentaje de la población económicamente activa que está desempleada: en general, sin empleo, disponible para trabajar y buscando activamente, según los criterios del organismo estadístico. Es un indicador clave de la salud del mercado laboral y tiene implicaciones sociales y económicas significativas.

Una tasa de desempleo baja generalmente indica un mercado laboral fuerte y que las personas pueden encontrar trabajo fácilmente. Sin embargo, un desempleo muy bajo (cercano a cero) es prácticamente imposible de lograr, pues siempre hay fricción en el mercado (personas cambiando de trabajo, nuevos entrantes, etc.). Una tasa de desempleo alta refleja dificultades económicas: menos oportunidades, menor consumo, y potencialmente mayor pobreza.

Cómo se calcula

Tasa de Desempleo = (Personas desempleadas / Población económicamente activa) × 100

La población económicamente activa suma ocupados y desempleados. La edad mínima y las excepciones dependen de la metodología nacional. Ser estudiante, jubilado o dedicarse al hogar no excluye automáticamente a alguien: cuenta si trabaja o cumple los criterios de desempleo. Quien no trabaja ni busca activamente suele quedar fuera de la fuerza laboral. Véase la metodología de la BLS.

Ejemplo práctico del desempleo

Un país tiene:

Población total: 50 millones
Población económicamente activa: 25 millones
Personas con trabajo: 23.5 millones
Personas desempleadas (buscando activamente): 1.5 millones

Tasa de Desempleo = (1.5 / 25) × 100 = 6%

El 6 % corresponde a la fuerza laboral, no a toda la población. Para interpretarlo hay que mirar su evolución, la participación y la calidad del empleo.

Balanza Comercial: cuánto vende y cuánto compra un país

La balanza comercial mide la diferencia entre el valor de las exportaciones (bienes vendidos al exterior) y las importaciones (bienes comprados del exterior) en un período determinado. Es parte de la cuenta corriente de la balanza de pagos.

Una balanza comercial positiva (superávit) significa que el país exporta más de lo que importa. Una balanza comercial negativa (déficit) significa que importa más de lo que exporta. Ambas situaciones tienen implicaciones económicas importantes.

Cálculo de la balanza comercial

Balanza Comercial = Exportaciones - Importaciones

Un resultado positivo indica superávit comercial; uno negativo, déficit.

Ejemplo práctico de balanza comercial

Un país exporta bienes por valor de $300 mil millones e importa por $250 mil millones.

Balanza Comercial = $300 - $250 = +$50 mil millones (superávit)

Este superávit registra más exportaciones que importaciones de bienes. No implica por sí solo una entrada neta total de divisas ni mayores reservas: faltan servicios, rentas, transferencias y movimientos financieros.

Importancia de la balanza comercial

Un superávit comercial es generalmente visto positivamente porque significa que los productos del país son competitivos en el mercado global. Sin embargo, un superávit muy grande puede generar fricciones comerciales internacionales. Un déficit comercial significa que el país consume más de lo que produce internamente, lo cual puede ser sostenible si va acompañado de inversión extranjera. Países como Estados Unidos tienen déficit comerciales persistentes pero sostienen este modelo mediante la inversión extranjera y el dólar como moneda de reserva mundial.

Tasa de Interés: el freno y el acelerador del crédito

La tasa de interés es el costo del dinero: el precio que se paga por usar capital ajeno durante un período determinado, expresado como un porcentaje para un período especificado. Los bancos centrales fijan tasas de política que influyen en las demás; las tasas de mercado y de los préstamos también dependen del plazo, el riesgo, la competencia y la oferta y demanda de fondos.

Una tasa de interés baja incentiva el consumo y la inversión (es más barato endeudarse), pero puede generar inflación. Una tasa alta desalienta el endeudamiento y ayuda a frenar la inflación, pero también ralentiza el crecimiento. Los bancos centrales ajustan las tasas para mantener un equilibrio entre crecimiento e inflación.

Para una explicación más detallada sobre tasas de interés, tipos de interés, y su impacto en economías personales, te recomendamos leer nuestro artículo sobre Conceptos básicos de economía.

Importancia de la tasa de interés

Las tasas afectan hipotecas, préstamos para autos, tarjetas de crédito, y ahorros
Tasas altas benefician a ahorradores pero perjudican a deudores
Los inversores siguen de cerca cambios en tasas porque afectan la rentabilidad de inversiones

Índice de Confianza del Consumidor: el ánimo que anticipa el gasto

El índice de confianza del consumidor (ICC) mide el grado de optimismo o pesimismo que tienen los consumidores sobre la economía actual y futura. Se obtiene mediante encuestas en las que se pregunta a hogares sobre su situación económica, perspectivas de empleo, y disposición a hacer compras principales.

Un ICC alto indica que los consumidores tienen confianza en que sus ingresos se mantendrán o aumentarán, que pueden encontrar empleo, y que es buen momento para comprar. Un ICC bajo refleja incertidumbre: las personas reducen consumo y ahorran más como protección.

Importancia del ICC

El ICC es un indicador anticipado importante: cambios en confianza generalmente preceden cambios en el consumo real. Si el ICC cae significativamente, los economistas pueden anticipar una reducción futura en consumo, lo que ralentiza la economía. Por el contrario, un ICC creciente sugiere expansión económica próxima.

Ejemplo práctico del ICC

Un país calcula su ICC preguntando a 1,000 hogares sobre:

Situación económica actual (mejor, igual, peor)
Perspectivas para los próximos 12 meses
Disposición a comprar casa, auto, o hacer mejoras al hogar

Si el 55% tiene expectativas positivas, el 25% neutrales, y el 20% negativas, el índice podría calcularse como (55 - 20) = 35 puntos (en una escala simplificada). Si este índice sube de 35 a 42 en el siguiente trimestre, refleja mejora en confianza.

Deuda Pública: lo que se debe frente a lo que se produce

La deuda pública es la cantidad total de dinero que un gobierno debe a prestamistas, que pueden ser ciudadanos, instituciones financieras, o gobiernos extranjeros. Surge cuando los gobiernos gastan más dinero del que recaudan en impuestos, necesitando financiar el déficit mediante endeudamiento.

La deuda pública se puede medir en términos absolutos (cantidad total) o en términos relativos (como porcentaje del PIB). El segundo es más útil para comparaciones internacionales y para evaluar sostenibilidad.

Cálculo de la deuda pública

Ratio Deuda Pública / PIB = (Deuda Pública Total / PIB Anual) × 100

Este ratio indica qué tan grande es la deuda respecto a la producción económica anual.

Ejemplo de deuda pública

Un país tiene:

Deuda pública: $2 billones
PIB anual: $10 billones

Ratio Deuda / PIB = ($2 / $10) × 100 = 20%

Un ratio del 20 % describe este ejemplo, pero no demuestra sostenibilidad. También importan intereses, vencimientos, moneda de denominación, ingresos fiscales, crecimiento y acceso al financiamiento. No existe un umbral único que clasifique la deuda de todos los países como segura o insostenible.

Importancia de la deuda pública

Una deuda moderada es normal y hasta beneficiosa: permite financiar inversión pública (infraestructura, educación). Pero deuda excesiva reduce la flexibilidad fiscal del gobierno, aumenta gastos en intereses, y puede llevar a crisis de refinanciamiento si los prestamistas pierden confianza. En casos extremos, un gobierno puede tener que reestructurar su deuda (por ejemplo, extender plazos o reducir pagos pactados) o incumplir. Una reestructuración no equivale simplemente a refinanciar a un interés menos favorable.

Tipo de Cambio: quién gana cuando una moneda se mueve

El tipo de cambio es el precio de una moneda en términos de otra. Por ejemplo, 1 dólar estadounidense = 0,95 euros, una cotización hipotética de 0,95 EUR (Euro (código ISO 4217)) por USD (United States dollar (dólar estadounidense, código ISO 4217)). En la convención de pares de mercado se escribiría USD/EUR = 0,95; EUR/USD indicaría cuántos dólares compra un euro. Los tipos de cambio fluctúan constantemente en mercados abiertos basándose en oferta y demanda de monedas.

El tipo de cambio es crucial para países que comercian internacionalmente. Una moneda fuerte (aprecia respecto a otras) significa que es más cara para extranjeros comprar productos del país, haciendo sus exportaciones menos competitivas. Una moneda débil (deprecia) abarata las exportaciones pero encarece las importaciones.

Importancia del tipo de cambio

Para exportadores: Una moneda débil favorece sus ventas al exterior
Para importadores: Una moneda fuerte hace más barata la compra de productos extranjeros
Para inversores extranjeros: Necesitan considerar riesgo de cambio
Para ahorradores: La apreciación puede mejorar la capacidad de comprar bienes extranjeros, pero no garantiza preservar el poder de compra interno

Ejemplo práctico del tipo de cambio

Un exportador de textiles vende productos por $1 millón USD. Si el tipo de cambio es 1 USD = 20 pesos, recibe 20 millones de pesos. Si la moneda se deprecia a 1 USD = 25 pesos (moneda más débil), recibe 25 millones de pesos por la misma venta. La moneda débil beneficia al exportador en este caso.

Ningún indicador cuenta solo la historia completa

Los indicadores económicos son el sistema de diagnóstico de la salud de una economía. Ninguno por sí solo cuenta toda la historia: un PIB que crece junto con inflación alta no es necesariamente una buena noticia. Un desempleo bajo acompañado de deuda pública insostenible tampoco lo es.

Un análisis económico completo requiere observar múltiples indicadores en conjunto, entender sus interrelaciones, y contextualizar los datos dentro de la historia y realidad política de cada país. Por ejemplo:

Si el PIB crece pero la balanza comercial empeora, el crecimiento podría estar basado en consumo insostenible de importaciones
Si el desempleo cae pero los salarios reales caen debido a inflación, la calidad de vida real puede estar deteriorándose
Si la deuda pública crece más rápido que el PIB, el país puede estar en una trayectoria insostenible

Por eso, la próxima vez que veas un titular con una sola cifra, pregúntate qué indicador falta para completar la historia.

Referencias

Banco Mundial (World Bank): https://www.worldbank.org/
- Proporciona datos sobre PIB, comercio, inflación, y desarrollo de todos los países
Fondo Monetario Internacional (FMI): https://www.imf.org/en/home
- Análisis económicos y bases de datos sobre indicadores macroeconómicos globales
Investopedia - Economic Indicators: https://www.investopedia.com/
- Definiciones detalladas y ejemplos prácticos de indicadores económicos
OCDE (Organización para la Cooperación y Desarrollo Económicos): https://www.oecd.org/
- Estadísticas y análisis de economías avanzadas
Curso de Indicadores Económicos https://platzi.com/cursos/indicadores-economicos/
- Curso en español que explica en detalle cada indicador y su interpretación

Building a Production-Ready Remote MCP Server

Sun, 22 Mar 2026 00:00:00 GMT

The Model Context Protocol (MCP) is an open standard for connecting LLMs (Large Language Models) to external tools and data sources via a JSON-RPC (JavaScript Object Notation Remote Procedure Call) interface. The key insight is that MCP gives you a proper, discoverable API (Application Programming Interface): a model can list available tools, inspect input schemas and optional output schemas, and request tool calls through a host application. MCP defines separate primitives for tools, resources, and prompts; retrieving a prompt is not itself execution of a function.

I was in charge of creating a remote MCP server to allow our customers to use LLMs to access the data they already have on our platform. What seemed straightforward turned into a deep dive into protocol design, session management, and authorization at scale.

Session Problems in Our April 2025 Deployment

There was one problem: in April 2025, the Python MCP ecosystem was still very young. I needed a library that I could embed directly in a Starlette/FastAPI application — one that was lightweight, type-safe, and simple to reason about. The official SDK (Software Development Kit) had significant issues with our infrastructure, particularly around session management in horizontally scaled environments:

Image: GitHub discussion about MCP session loss in horizontally scaled environments

GitHub Discussion Link

Rather than working around these limitations, I decided to build a library from scratch with the characteristics mentioned above — and without the problems encountered in that deployment. This is a historical account, not a claim that current SDK releases require the same workaround. The official SDK now documents deployment and scaling options, including stateless operation.

Building http-mcp

http-mcp (published on PyPI (Python Package Index) as http-mcp) provides tools and prompts over HTTP (Hypertext Transfer Protocol) and stdio. Its supported features and protocol versions should be checked against the client in use. I designed it around a few principles:

1. Pydantic-first, schema automatically generated. Tools are declared by wrapping a Python function in a Tool dataclass and pointing at its Pydantic input/output models. The JSON (JavaScript Object Notation) schema that MCP clients use for discovery is derived automatically from those models:

from http_mcp.types import Arguments, Tool
from pydantic import BaseModel, Field, UUID4


class FixMetadataInput(BaseModel):
    vulnerability_id: UUID4 = Field(description="Vulnerability uuid4 id")


class FixMetadataOutput(BaseModel):
    version: str = Field(
        description="Safe version of the dependency",
    )
    is_breaking_change: bool = Field(
        description="Whether upgrading to safe version is a breaking change",
    )


def get_fix_metadata(
    arguments: Arguments[FixMetadataInput],
) -> FixMetadataOutput:
    return FixMetadataOutput(
        version="2.3.4",
        is_breaking_change=False,
    )


tool = Tool(
    inputs=FixMetadataInput,
    output=FixMetadataOutput,
    func=get_fix_metadata,
)

With inputs as Pydantic models validation happens automatically before the data reaches the tool function. Inputs are rejected with structured errors if they do not match the schema. Another advantage is that every field is self-documenting — by using Field(description="..."), both the input and output schemas carry human-readable documentation that MCP clients can surface to users and models alike.

2. Starlette-native auth. Rather than inventing a new auth model, the library plugs into Starlette's AuthenticationMiddleware. Scope-based access control just works: tools declare which scopes they require, and the framework filters them before listing or invocation.

3. Tool errors as model feedback. The http-mcp documentation describes return_error_message=True as converting ToolInvocationError exceptions into ErrorMessage results. This is a library feature, not a general guarantee about all Pydantic failures or HTTP status codes. A useful error can explain what the caller should correct, provided the client exposes that result to the model.

Tool(
    inputs=FixMetadataInput,
    output=FixMetadataOutput,
    func=get_fix_metadata,
    return_error_message=True,  # Return supported tool invocation errors as ErrorMessage results
    scopes=("mcp:read",),
)

This can make recoverable failures actionable. Whether a retry succeeds, or saves tokens, depends on the error and the client.

Stateful Sessions: A Scaling Problem

The early versions of the MCP Python SDK maintained session state across requests. This creates a critical problem in horizontally scaled environments: if your MCP server runs on multiple workers behind a load balancer, each request might hit a different worker instance. Session data stored on Worker A is invisible to Worker B, causing a session lookup to fail. That can interrupt tool access, but it does not inherently erase the model's conversation history, which the host manages.

The difficulty is coordinating state, not the mere existence of state:

Requests can reach a worker that does not own an in-memory session.
Session affinity is one option, with failover trade-offs. Shared state and routing to the owning worker are other possible designs.
Stateless request handling simplifies horizontal scaling, but stateful services can also scale with appropriate coordination and recovery.

For our workload, avoiding per-worker session state was a useful trade-off.

The solution was to implement a stateless HTTP transport that treats each request as independent. Instead of maintaining session objects in memory, the transport sends all necessary context in each request. This stateless design became the foundation for http-mcp: https://pypi.org/project/http-mcp/

Figure: With the session in worker memory, the second request fails as soon as the balancer picks another worker. When each request carries its context, any worker can answer. (interactive version in the post)

Remote MCP Architecture: Streamable HTTP vs stdio

The 2025-11-25 MCP specification defines two standard transports and permits custom ones:

Stdio Transport: The MCP server runs as a subprocess on the same machine, communicating via standard input/output. This is simple for local integrations but limits you to a single host.
Streamable HTTP: The MCP server runs as a separate service (often remote), communicating over HTTP. This enables true separation of concerns and horizontal scaling.

For production systems, HTTP transport is the practical choice. It allows separate deployment and scaling of the MCP server, easier monitoring and logging, integration with load balancers and service meshes, and language-agnostic communication between components.

The trade-off is complexity: handle network errors, timeouts, authentication, and any chosen session lifecycle. Streamable HTTP permits optional sessions; it does not require every server to be stateless.

Challenge 1: Errors the Caller Can Act On

Input-schema validation and tool execution are different stages. Malformed protocol requests can produce JSON-RPC errors; recoverable execution failures can be represented as tool results. MCP defines an isError flag for tool execution errors. See the tool error rules.

In http-mcp, the documented return_error_message option handles supported ToolInvocationError exceptions. For example, a handler can report that a requested record is unavailable, with enough context for the caller to change its arguments. Do not assume that every client displays errors identically or that a retry will succeed.

Figure: A recoverable tool error can guide a revised request when the client passes its details to the model. The outcome depends on the client and the error. (interactive version in the post)

Challenge 2: Context-Based Tool Exposure

Not all tools should be available to all callers. You might want:

Public tools (no auth required): A "get_server_status" tool anyone can use
Private tools (auth required): A "delete_user" tool only admins can access
Scoped tools: Different tools for different contexts or user roles

I solved this by leveraging Starlette's scope system. Each HTTP request carries scope metadata (user, role, permissions), and tools declare which scopes they require. With the same models as before, the only addition is scopes:

Tool(
    inputs=FixMetadataInput,
    output=FixMetadataOutput,
    func=get_fix_metadata,
    scopes=("mcp:read",),  # Only callers with "mcp:read" scope can use this tool
)

This approach has several benefits: tools are self-documenting about their requirements, authorization is centralized and consistent, you can change permissions without modifying tool code, and the AI (Artificial Intelligence) agent can query which tools are available in its current context — which reduces suggestions for unavailable tools. A client can still submit a hidden or stale tool name, so the server must authorize every invocation.

Figure: Scope filtering limits discovery. The server must also reject unauthorized tools/call requests, even if the client knows a hidden tool name. (interactive version in the post)

Lessons Learned

Building a production MCP server taught me several things:

Stateless by default. Design your server assuming instances are ephemeral. This simplifies request routing and avoids some session-related bugs.
Error messages are feedback loops. The AI agent can adjust subsequent calls using error messages. Make them helpful and specific — generic errors cause retries and wasted tokens. Some MCP clients do not even pass error responses to the LLM; supported tool errors can instead be returned as results via return_error_message=True.
Explicit permissions matter. Making scope requirements visible in tool definitions makes it easier to reason about security and to conditionally expose tools and prompts based on the caller's identity.

References

Official MCP Specification: modelcontextprotocol.io
My HTTP Transport Implementation: http-mcp on PyPI
Stateless Sessions Discussion: Python SDK GitHub Issue #520

SOLID Python

Sun, 22 Mar 2026 00:00:00 GMT

The SOLID (Single responsibility, Open/closed, Liskov substitution, Interface segregation, Dependency inversion) principles are five design guidelines that help developers write code that is more understandable, flexible, and maintainable. Popularized by Robert C. Martin, these principles bring together ideas from several authors and are applicable across programming languages, including Python. This article explores each principle with practical Python examples that demonstrate how to apply them in real-world scenarios.

SRP: Single Responsibility Principle

A class should have only one reason to change.

A reason to change corresponds to a responsibility to an actor or stakeholder group. It does not mean a class must have only one method. This principle helps to keep classes small, focused, and maintainable. When a class has multiple responsibilities, changes to one responsibility can inadvertently break another, making the code harder to test and understand.

Consider a class that handles both user authentication and sending email notifications. If the authentication logic needs to change, and the email logic also needs to change, the class becomes a point of conflict. By separating these concerns into distinct classes, each can evolve independently.

Here's a before-and-after example. The password comparison is a deliberately simplified stand-in for authentication, not a password-storage design. Real applications should use a password-hashing library's verification function and store salted password hashes, as described in the OWASP (Open Worldwide Application Security Project) Password Storage Cheat Sheet.

# BEFORE: Violates SRP
class User:
    def __init__(self, email: str, password: str):
        self.email = email
        self.password = password

    def authenticate(self, provided_password: str) -> bool:
        # Authentication logic
        return self.password == provided_password

    def send_welcome_email(self):
        # Email logic mixed with user logic
        print(f"Sending welcome email to {self.email}")
        return True


# AFTER: Follows SRP
class User:
    def __init__(self, email: str, password: str):
        self.email = email
        self.password = password

    def authenticate(self, provided_password: str) -> bool:
        return self.password == provided_password


class EmailService:
    @staticmethod
    def send_welcome_email(user: User):
        print(f"Sending welcome email to {user.email}")
        return True

The refactored version gives each class a single, clear responsibility. User handles authentication, while EmailService handles email operations. This makes testing easier (you can mock the email service) and allows either responsibility to change independently.

Why it matters: Single responsibility makes code easier to test, reuse, and modify without unintended side effects.

OCP: Open/Closed Principle

Software entities should be open for extension but closed for modification.

This principle encourages designing systems so that new functionality can be added without changing existing code. Instead of modifying a class to support new behavior, you extend it through inheritance, composition, or abstraction. This reduces the risk of breaking existing functionality and makes codebases more stable.

In Python, the Open/Closed Principle is elegantly expressed through abstract base classes (ABC) and polymorphism. By defining an interface (abstract class), you allow different implementations to be "plugged in" without modifying the core logic that uses them.

Consider a payment processing system that needs to support multiple payment methods:

from abc import ABC, abstractmethod


# BEFORE: Violates OCP - requires modification for each new payment type
class PaymentProcessor:
    def process(self, payment_type: str, amount: float) -> bool:
        if payment_type == "credit_card":
            return self._process_credit_card(amount)
        elif payment_type == "paypal":
            return self._process_paypal(amount)
        elif payment_type == "bitcoin":
            return self._process_bitcoin(amount)
        else:
            raise ValueError(f"Unknown payment type: {payment_type}")

    def _process_credit_card(self, amount: float) -> bool:
        print(f"Processing credit card payment of ${amount}")
        return True

    def _process_paypal(self, amount: float) -> bool:
        print(f"Processing PayPal payment of ${amount}")
        return True

    def _process_bitcoin(self, amount: float) -> bool:
        print(f"Processing Bitcoin payment of ${amount}")
        return True


# AFTER: Follows OCP - extensible without modification
class PaymentMethod(ABC):
    @abstractmethod
    def process(self, amount: float) -> bool:
        pass


class CreditCardPayment(PaymentMethod):
    def process(self, amount: float) -> bool:
        print(f"Processing credit card payment of ${amount}")
        return True


class PayPalPayment(PaymentMethod):
    def process(self, amount: float) -> bool:
        print(f"Processing PayPal payment of ${amount}")
        return True


class BitcoinPayment(PaymentMethod):
    def process(self, amount: float) -> bool:
        print(f"Processing Bitcoin payment of ${amount}")
        return True


class PaymentProcessor:
    def __init__(self, payment_method: PaymentMethod):
        self.payment_method = payment_method

    def process(self, amount: float) -> bool:
        return self.payment_method.process(amount)


# Using it:
processor = PaymentProcessor(CreditCardPayment())
processor.process(100.00)

# To add a new payment method, simply create a new class:
class ApplePayPayment(PaymentMethod):
    def process(self, amount: float) -> bool:
        print(f"Processing Apple Pay payment of ${amount}")
        return True

processor = PaymentProcessor(ApplePayPayment())
processor.process(50.00)

The refactored version is closed for modification—the PaymentProcessor class doesn't change when you add new payment methods—and open for extension through the PaymentMethod abstraction. This follows the Strategy pattern, a classic way to implement OCP.

Why it matters: Open/Closed Principle reduces regression risks and allows your codebase to grow without constant refactoring of existing code.

LSP: Liskov Substitution Principle

Subtypes must be substitutable for their base types without breaking the application.

This principle ensures that derived classes can be used in place of their base classes without causing errors or unexpected behavior. If a function expects a base class, passing a derived class should work seamlessly. Violating LSP typically results in type checking or conditional logic that defeats the purpose of polymorphism.

A classic example involves geometric shapes. While a square is technically a rectangle mathematically, in object-oriented design, if Rectangle allows independent width and height changes but Square requires them to be equal, substituting a Square for a Rectangle will cause problems.

# BEFORE: Violates LSP
class Rectangle:
    def __init__(self, width: float, height: float):
        self._width = width
        self._height = height

    def set_width(self, width: float):
        self._width = width

    def set_height(self, height: float):
        self._height = height

    def get_area(self) -> float:
        return self._width * self._height


class Square(Rectangle):
    def set_width(self, width: float):
        self._width = width
        self._height = width  # Enforce square property

    def set_height(self, height: float):
        self._width = height
        self._height = height  # Enforce square property


# This breaks LSP:
def calculate_area(shape: Rectangle) -> float:
    shape.set_width(5)
    shape.set_height(4)
    expected_area = 20
    actual_area = shape.get_area()
    assert actual_area == expected_area  # This fails if shape is a Square!
    return actual_area


# AFTER: Follows LSP
from abc import ABC, abstractmethod


class Shape(ABC):
    @abstractmethod
    def get_area(self) -> float:
        pass


class Rectangle(Shape):
    def __init__(self, width: float, height: float):
        self._width = width
        self._height = height

    def set_width(self, width: float):
        self._width = width

    def set_height(self, height: float):
        self._height = height

    def get_area(self) -> float:
        return self._width * self._height


class Square(Shape):
    def __init__(self, side: float):
        self._side = side

    def set_side(self, side: float):
        self._side = side

    def get_area(self) -> float:
        return self._side ** 2


def calculate_area(shape: Shape) -> float:
    area = shape.get_area()
    return area


# Now both can be used safely:
rect = Rectangle(5, 4)
rect.set_width(5)
rect.set_height(4)
assert rect.get_area() == 20

square = Square(5)
square.set_side(5)
assert square.get_area() == 25

In the corrected version, Rectangle and Square are both concrete implementations of Shape. They don't try to substitute for each other inappropriately; each is designed to work correctly with its own interface. This allows polymorphic usage without breaking contracts.

Why it matters: LSP ensures that inheritance hierarchies are logically sound and that polymorphism works as intended without surprises.

ISP: Interface Segregation Principle

Clients should not depend on interfaces they do not use.

This principle suggests creating many focused, client-specific interfaces rather than one monolithic interface. If a class is forced to implement methods it doesn't need, the implementation becomes bloated and changes in unrelated methods can affect unrelated clients.

In Python, this is best expressed using abstract base classes (ABC) or Protocols that define narrow, specific contracts. Each implementation only depends on the methods it actually uses.

# BEFORE: Violates ISP
from abc import ABC, abstractmethod


class Worker(ABC):
    @abstractmethod
    def work(self):
        pass

    @abstractmethod
    def eat_lunch(self):
        pass


class HumanWorker(Worker):
    def work(self):
        print("Human working...")

    def eat_lunch(self):
        print("Human eating lunch...")


class RobotWorker(Worker):
    def work(self):
        print("Robot working...")

    def eat_lunch(self):
        # Robots don't eat! But they're forced to implement this.
        raise NotImplementedError("Robots don't eat")


# AFTER: Follows ISP
class Workable(ABC):
    @abstractmethod
    def work(self):
        pass


class Eatable(ABC):
    @abstractmethod
    def eat_lunch(self):
        pass


class HumanWorker(Workable, Eatable):
    def work(self):
        print("Human working...")

    def eat_lunch(self):
        print("Human eating lunch...")


class RobotWorker(Workable):
    def work(self):
        print("Robot working...")
    # No need to implement eat_lunch


# Using them:
def start_work(worker: Workable):
    worker.work()


def lunch_break(eater: Eatable):
    eater.eat_lunch()


human = HumanWorker()
start_work(human)
lunch_break(human)

robot = RobotWorker()
start_work(robot)
# lunch_break(robot)  # Type error - robot is not Eatable, which is correct!

By segregating the Worker interface into smaller, focused interfaces (Workable and Eatable), each class only implements what it needs. This prevents robot implementations from being forced to handle eating, and it makes the code's intent clearer.

Why it matters: Interface Segregation keeps implementations lean and prevents classes from being burdened with methods they don't use.

DIP: Dependency Inversion Principle

High-level modules should depend on abstractions, not low-level modules. Both should depend on abstractions.

This principle reverses the typical dependency direction in traditional layered architectures. Instead of high-level business logic depending on low-level utility classes, both depend on shared abstractions. This reduces coupling and makes swapping implementations easier, especially for testing.

Dependency injection is the primary technique for implementing DIP in Python. Rather than a class creating its own dependencies, they're provided (injected) from outside.

# BEFORE: Violates DIP
class EmailService:
    def send(self, contact: str, message: str):
        print(f"Sending email to {contact}: {message}")


class SMSService:
    def send(self, contact: str, message: str):
        print(f"Sending SMS to {contact}: {message}")


class Notifier:
    def __init__(self):
        # High-level module depends directly on low-level modules
        self.email_service = EmailService()
        self.sms_service = SMSService()

    def notify_user(self, user_type: str, contact: str, message: str):
        if user_type == "email":
            self.email_service.send(contact, message)
        elif user_type == "sms":
            self.sms_service.send(contact, message)


# AFTER: Follows DIP
from abc import ABC, abstractmethod


class NotificationService(ABC):
    @abstractmethod
    def send(self, contact: str, message: str):
        pass


class EmailService(NotificationService):
    def send(self, contact: str, message: str):
        print(f"Sending email to {contact}: {message}")


class SMSService(NotificationService):
    def send(self, contact: str, message: str):
        print(f"Sending SMS to {contact}: {message}")


class PushNotificationService(NotificationService):
    def send(self, contact: str, message: str):
        print(f"Sending push notification to {contact}: {message}")


class Notifier:
    def __init__(self, service: NotificationService):
        # Depend on abstraction, not concrete class
        self.service = service

    def notify_user(self, contact: str, message: str):
        self.service.send(contact, message)


# Using it:
email_notifier = Notifier(EmailService())
email_notifier.notify_user("user@example.com", "Hello!")

sms_notifier = Notifier(SMSService())
sms_notifier.notify_user("+1234567890", "Hello!")

push_notifier = Notifier(PushNotificationService())
push_notifier.notify_user("device123", "Hello!")


# Testing is now easy - inject a mock:
class MockNotificationService(NotificationService):
    def __init__(self):
        self.messages = []

    def send(self, contact: str, message: str):
        self.messages.append((contact, message))


mock_service = MockNotificationService()
test_notifier = Notifier(mock_service)
test_notifier.notify_user("test@example.com", "Test message")
assert len(mock_service.messages) == 1

The refactored version inverts dependencies. The Notifier class (high-level module) doesn't create or depend on concrete EmailService or SMSService; instead, it depends on the NotificationService abstraction. This allows you to inject any NotificationService implementation, making the code flexible and testable.

Figure: Before, Notifier points down at the concrete services it creates. After, every arrow ends at NotificationService: Notifier uses it, and each service implements it, so adding PushNotificationService does not touch Notifier. (interactive version in the post)

Why it matters: Dependency Inversion decouples high-level business logic from low-level implementation details, making your code more testable, flexible, and resilient to change.

Applying SOLID Together

These five principles work best as a cohesive set. A project that follows all of them tends to be:

Modular: Each class has a clear, focused purpose (SRP).
Extensible: New features can be added without modifying existing code (OCP).
Predictable: Derived classes behave as their base classes expect (LSP).
Focused: Classes only depend on methods they use (ISP).
Decoupled: High-level logic doesn't depend on implementation details (DIP).

While strict adherence to SOLID isn't always necessary for small scripts or prototypes, investing in these principles early pays dividends in larger, evolving codebases. Python's flexibility and support for abstractions (ABC, Protocols, duck typing) make implementing SOLID principles straightforward.

References

Python Software Foundation. "abc — Abstract Base Classes." Python Documentation. https://docs.python.org/3/library/abc.html
Curso de Patrones de Diseño y SOLID en Python. Platzi. https://platzi.com/cursos/solid-python/

Time Value of Money: Why a Dollar Today Beats a Dollar Tomorrow

Sun, 22 Mar 2026 00:00:00 GMT

Save $500 a month for 30 years at 8%, and you end up with about $745,180, even though you only deposited $180,000. Borrow $300,000 for 30 years at 5%, and you pay about $279,767 in interest on top of the loan. Both numbers come from the same idea: with a positive rate of return, a dollar today is worth more than the same dollar received later. That idea is the time value of money, and it underlies nearly every financial decision, from personal savings goals to corporate investments.

Money has a time value for three reasons. First, money can earn returns: a dollar invested today at 5% a year grows to $1.05 a year from now. Second, inflation erodes purchasing power, so future dollars buy less than today's. Third, the future is uncertain: a promised payment next year may not arrive.

This article walks through the formulas behind those two opening numbers and the decisions they support: choosing between a lump sum and installments, evaluating an investment, planning for retirement, and comparing mortgages. All rates and cash flows below are hypothetical, constant unless stated otherwise, and exclude fees and taxes. See OpenStax Principles of Finance, time value of money.

Compound interest earns interest on interest

When you invest, you earn interest on the principal. Over time, the interest you earn also earns interest. This process of earning returns on your returns is called compounding, and it makes growth exponential.

The fundamental formula for compound interest is:

FV = PV × (1 + r)^n

Where:

FV is the future value of the investment
PV is the present value (initial amount) of the investment
r is the interest rate per period (expressed as a decimal)
n is the number of periods

For example, if you invest $1,000 at a 6% annual interest rate for 10 years, your future value is:

FV = 1000 × (1.06)^10 = 1000 × 1.791 = $1,791

Your investment grew by almost 80%. The whole $791 gain is interest, and $191 of it is interest earned on earlier interest.

We can rearrange this formula to work backwards. If you know the future value of an investment and want its present value, use the present value formula:

PV = FV / (1 + r)^n

This tells you how much money you need to invest today to reach a specific financial goal in the future.

The gap between simple and compound interest widens with time

To see what compounding adds, compare it with simple interest. With simple interest, you earn returns only on the original principal. The formula is:

FV = PV × (1 + r × n)

Using the previous example, $1,000 at 6% simple interest for 10 years grows to:

FV = 1000 × (1 + 0.06 × 10) = 1000 × 1.6 = $1,600

With compound interest, you earn returns on the principal and on all accumulated interest:

FV = 1000 × (1.06)^10 = $1,791

The difference is $191 in favor of compound interest. Simple interest adds the same $60 every year, while compound interest adds a little more each year, so the gap widens with longer periods and higher rates. After 30 years, simple interest reaches $2,800 and compound interest reaches $5,743.

Figure: $1,000 at 6% a year. Simple interest adds the same $60 every year; compound interest adds more each year, so compounding is worth $191 after 10 years and $2,943 after 30. (interactive version in the post)

Savings accounts and most investments compound, but loans apply interest in different ways. Loan interest may accrue on the outstanding balance without unpaid interest being capitalized. Check the actual contract rather than assuming every product uses the same formula.

Every cash flow is a lump sum, an annuity, or an irregular series

Financial situations involve three main types of cash flows: lump sums, annuities, and irregular series of cash flows. Each requires its own valuation approach.

Lump sum

A lump sum is a single amount of money received or paid at one point in time. It uses the two formulas above: FV = PV × (1 + r)^n to move it forward and PV = FV / (1 + r)^n to bring it back.

College savings example

Suppose you want to save for your child's college education and estimate you'll need $150,000 in 15 years. If you can invest in a fund earning 7% annually, how much do you need to deposit today?

PV = 150,000 / (1.07)^15 = 150,000 / 2.759032 = $54,367

You would need to deposit approximately $54,367 today to reach your goal of $150,000 in 15 years, assuming a consistent 7% annual return.

Annuity

An annuity is a series of equal payments made at regular intervals over time. Common examples include mortgage payments, retirement withdrawals, and savings contributions.

The future value of an ordinary annuity (payments at the end of each period) is:

FV = PMT × ((1 + r)^n - 1) / r

The present value of an ordinary annuity is:

PV = PMT × (1 - (1 + r)^(-n)) / r

Where:

FV is the future value of the annuity
PV is the present value of the annuity
PMT is the payment amount per period
r is the interest rate per period
n is the number of periods

Monthly savings plan example

You decide to save $500 per month for retirement in an investment account with a hypothetical nominal annual rate of 8%, compounded monthly. How much will you have accumulated after 30 years?

Use the exact monthly rate r = 0.08 / 12 (approximately 0.6667%). If 8% were an effective annual rate instead, the monthly rate would be (1.08)^(1/12) - 1. These are different assumptions.

Number of periods: 30 years × 12 months = 360

r = 0.08 / 12
FV = 500 × ((1 + r)^360 - 1) / r ≈ $745,179.72

By saving $500 per month for 30 years, you accumulate approximately $745,179.72. Your total contributions are $500 × 360 = $180,000, so $565,179.72 comes from investment returns: more than three times what you put in.

The formula also runs in reverse. If you want $500,000 at retirement and need to know the monthly payment, solve for PMT:

PMT = FV × r / ((1 + r)^n - 1)
PMT = 500,000 × r / ((1 + r)^360 - 1) ≈ $335.49

At the same 8% rate, about $335.49 a month for 30 years reaches $500,000.

Series of cash flows and net present value

Many real-world investments involve irregular cash flows: different amounts at different times. To evaluate these, we use net present value (NPV), which discounts every future cash flow back to the present and sums them.

The NPV formula is:

NPV = Σ(t=0..n) CF_t / (1 + r)^t

Where:

CF_t is the cash flow in period t
r is the discount rate (required rate of return)
t is the time period
n is the total number of periods

A positive NPV means the modeled cash flows exceed the required return under the chosen assumptions. For mutually exclusive, comparable projects without capital constraints, higher NPV is preferred. Risk, unequal project lifetimes and funding constraints can require additional analysis.

Investment project evaluation example

Your company is considering a project with the following cash flows over 5 years. The required rate of return is 10%.

Year 0: -$100,000 (initial investment)
Year 1: $30,000
Year 2: $35,000
Year 3: $40,000
Year 4: $25,000
Year 5: $20,000

Calculate the NPV:

NPV = -100,000 / 1.10^0 + 30,000 / 1.10^1
    + 35,000 / 1.10^2 + 40,000 / 1.10^3
    + 25,000 / 1.10^4 + 20,000 / 1.10^5

NPV ≈ -100,000 + 27,272.73 + 28,925.62 + 30,052.59
      + 17,075.34 + 12,418.43 = $15,744.70

The modeled NPV is $15,744.70, positive at the assumed 10% discount rate. The project returns $150,000 in total, but in today's money those returns are worth only $115,744.70. A positive NPV supports accepting the project if the forecasts, risk adjustment and other constraints are appropriate; it is not a guarantee of the actual outcome.

Inflation shrinks what your returns can buy

When evaluating investments, distinguish between nominal returns and real returns.

Nominal returns are the stated, unadjusted returns. If your savings account earns 2% annually, that's the nominal rate.

Real returns account for inflation. If the nominal rate is 2% but inflation is 3%, your real return is negative: your purchasing power declined.

The exact purchasing-power adjustment for the same period is:

(1 + r_real) = (1 + r_nominal) / (1 + i)

Where i is the inflation rate. The commonly used approximation is real rate ≈ nominal rate − inflation.

For example, if your investment earns 6% (nominal) and inflation is 2%, your real return is:

r_real = (1.06) / (1.02) - 1 = 0.0392 = 3.92%

Your purchasing power increased by about 3.92%, not 6%. Over long periods, the gap between nominal and real returns compounds too. To build real wealth, you need returns that exceed inflation.

The same formulas price mortgages and retirement

Mortgage payments

When you take out a mortgage, the lender uses the present value of an annuity formula to calculate your monthly payment. Take a $300,000 mortgage for 30 years at a nominal annual rate of 5%, compounded monthly: r = 0.05 / 12, with 360 end-of-month payments. This excludes fees, insurance and taxes.

PMT = 300,000 × r / (1 - (1 + r)^(-360)) ≈ $1,610.46

Using the unrounded payment, total principal and interest is approximately $579,767.35, including $279,767.35 in interest: almost as much again as the loan itself. Actual cent-rounded payments require a small final-payment adjustment. Because interest accrues on the outstanding balance, paying extra principal early shrinks the balance on which every later month's interest is charged.

Retirement planning

Retirement planning is fundamentally a time value of money problem. You need to estimate:

How much you'll need annually in retirement
How many years you'll need the money
What return your investments will earn
How much you need to save now

If you need $50,000 annually in retirement for 30 years and expect 6% returns, the present value of your retirement needs is:

PV = 50,000 × (1 - (1.06)^(-30)) / 0.06 ≈ $688,241.56

Under this simplified model, the initial amount is $688,241.56: less than half of the $1,500,000 you will withdraw, because the balance keeps earning while you draw it down. It assumes fixed nominal withdrawals at year-end and a constant 6% return, with no taxes, fees or inflation increases. Variable returns introduce sequence-of-returns risk, so this is not a safe-withdrawal guarantee. Working backward with the annuity formula, you can determine how much to save monthly to reach this goal.

Run the calculations in Python

These functions reproduce the examples above. Change the inputs to model your own scenarios:

def future_value(pv, rate, periods):
    """Calculate future value of a lump sum."""
    return pv * (1 + rate) ** periods

def present_value(fv, rate, periods):
    """Calculate present value of a future amount."""
    return fv / (1 + rate) ** periods

def annuity_future_value(pmt, rate, periods):
    """Calculate future value of an ordinary annuity."""
    if rate == 0:
        return pmt * periods
    return pmt * (((1 + rate) ** periods - 1) / rate)

def annuity_present_value(pmt, rate, periods):
    """Calculate present value of an ordinary annuity."""
    if rate == 0:
        return pmt * periods
    return pmt * ((1 - (1 + rate) ** (-periods)) / rate)

def net_present_value(cash_flows, discount_rate):
    """Calculate NPV of a series of cash flows."""
    npv = 0
    for t, cf in enumerate(cash_flows):
        npv += cf / (1 + discount_rate) ** t
    return npv

# Example: College savings, 150,000 needed in 15 years at 7%
deposit = present_value(150_000, 0.07, 15)
print(f"Deposit needed today: ${deposit:,.2f}")

# Example: End-of-month deposits; nominal 8% annually, compounded monthly
monthly_rate = 0.08 / 12
months = 30 * 12
monthly_payment = 500

fv = annuity_future_value(monthly_payment, monthly_rate, months)
print(f"Future value of retirement savings: ${fv:,.2f}")

# Example: Evaluate a project
project_cashflows = [-100000, 30000, 35000, 40000, 25000, 20000]
npv = net_present_value(project_cashflows, 0.10)
print(f"Net present value of project: ${npv:,.2f}")

Move every cash flow to the same date before you compare

Money today has options. You can spend it, invest it, or lend it and earn returns. Future money offers no such flexibility, because it is locked in the future. Time value calculations translate future cash flows into their equivalent present value so you can compare opportunities fairly.

The formulas are only as good as their inputs. Small changes in the interest rate, inflation, or time period can significantly alter the results, so treat them as guides and question your assumptions.

The practical rule is short: before you compare a lump sum with installments, two investments, or two loans, bring every cash flow to the same date at the same rate. Only then are you comparing like with like.

References

Brealey, R. A., Myers, S. C., & Allen, F. (2020). "Principles of Corporate Finance" (13th ed.). McGraw-Hill Education.
Investopedia. "Time Value of Money (TVM)."
Khan Academy. "Finance and Capital Markets."

JSON-RPC V2.0

Sun, 15 Mar 2026 00:00:00 GMT

This article is based on the JSON‑RPC 2.0 Specification by the JSON‑RPC (JavaScript Object Notation Remote Procedure Call) Working Group, Copyright (C) 2007–2010 by the JSON‑RPC Working Group, used under the permissions granted in the specification.

JSON-RPC is a stateless, lightweight remote procedure call (RPC) protocol.

Request Object

{
  "jsonrpc": "2.0",
  "method": "call_function",
  "params": {
    "name": "function_name",
    "arguments": {
      "user_name": "jondoe"
    }
  },
  "id": 2
}

jsonrpc: Version of the JSON-RPC protocol (String, required, must be "2.0").
method: Name of the method to be invoked (String, required).
params: Structured value that holds parameter values for invocation (Array or Object, optional).
id: Identifier defined by the client (String, Number, or Null, optional). If omitted, the message is assumed to be a notification. Otherwise, the server must reply with the same value in the response object.

Notifications

Messages with no id. A Request object that is a notification signifies the client's lack of interest in the Response object. The server must not reply to a notification.

Parameters Structure

Parameters must be provided as structured values, either by position through an array or by name through an object.

Response Object

The response is expressed as a single JSON object with the following members.

jsonrpc: Version of the JSON-RPC protocol (String, required, must be "2.0").
result: Required on success. It must not exist if there was an error invoking the method.
error: Required on error. It must not exist if there was no error during invocation.
id: Required. If there was an error detecting the id in the Request object, it must be null.

Either the result member or error member must be included, but not both.

Error Object

Object with the following members:

code: An integer that indicates the error type that occurred.
message: Description of the error (String). Concise single sentence.
data: A primitive or structured value containing additional information about the error (optional).

See the JSON-RPC error object specification.

Batch

The client sends an array filled with Request objects. The server responds with an array containing the corresponding Response objects. A response object should exist for each request object except for notification requests.

The responses may be returned in any order in the responses array. The server must not return an empty array and should return nothing at all for batches containing only valid notifications. An empty batch ([]) is invalid and receives a single Invalid Request error object, not an empty response array.

Figure: A batch of three requests gets two responses: the notification has no id, so nothing answers it, and the responses may come back in any order. A batch of only notifications gets no reply at all. (interactive version in the post)

Attribution and Copyright Notice

This article includes content derived from the JSON-RPC 2.0 specification.

This document and translations of it may be used to implement JSON-RPC, it may be copied and furnished to others, and derivative works that comment on or otherwise explain it or assist in its implementation may be prepared, copied, published and distributed, in whole or in part, without restriction of any kind, provided that the above copyright notice and this paragraph are included on all such copies and derivative works. However, this document itself may not be modified in any way.

The limited permissions granted above are perpetual and will not be revoked.

This document and the information contained herein is provided "AS IS" and ALL WARRANTIES, EXPRESS OR IMPLIED are DISCLAIMED, INCLUDING BUT NOT LIMITED TO ANY WARRANTY THAT THE USE OF THE INFORMATION HEREIN WILL NOT INFRINGE ANY RIGHTS OR ANY IMPLIED WARRANTIES OF MERCHANTABILITY OR FITNESS FOR A PARTICULAR PURPOSE.

Most Useful Git Commands

Sat, 14 Mar 2026 00:00:00 GMT

Git is one of those tools you use every day, but most people only scratch the surface. Here are the commands I find myself reaching for the most.

The Essentials

git log --oneline --graph --all

The default git log output is verbose. This combination gives you a compact, visual history of all branches.

git log --oneline --graph --all

You can alias this to something short like git lg by adding it to your .gitconfig:

git config --global alias.lg "log --oneline --graph --all"

git diff --staged

Before committing, I always check what is actually staged. git diff alone shows unstaged changes, but --staged shows exactly what will go into the next commit.

git diff --staged

git stash

When you need to quickly switch context without committing half-done work, stash saves your working directory and index state.

git stash
git checkout other-branch
# do some work
git checkout -
git stash pop

Naming stashes

By default stashes get a generic name based on the branch and commit. Use -m to give them a meaningful description so you can identify them later.

git stash push -m "wip: auth refactor"
git stash list
# stash@{0}: On main: wip: auth refactor

Stashing specific files

You do not have to stash everything. Use push with file paths to stash only certain files.

# stash only specific files
git stash push -m "just the config" src/config.ts src/env.ts

# stash only staged changes (leaves unstaged changes in the working tree)
git stash push --staged -m "staged only"

Stashing untracked files

By default git stash only saves tracked files. Use -u to include untracked files, or -a to also include ignored files.

# include untracked files
git stash push -u -m "with new files"

# include untracked AND ignored files
git stash push -a -m "everything"

Stashing interactively

Use -p (patch) to interactively choose which hunks to stash, similar to git add -p.

git stash push -p -m "partial changes"

Listing and inspecting stashes

# list all stashes
git stash list

# show the diff of a specific stash
git stash show stash@{1}

# show the full diff with file contents
git stash show -p stash@{1}

Applying and removing stashes

# apply the most recent stash; remove it only if application succeeds
git stash pop

# apply without removing (keeps the stash for reuse)
git stash apply

# apply a specific stash
git stash apply stash@{2}

# drop a specific stash
git stash drop stash@{1}

# clear all stashes
git stash clear

Creating a branch from a stash

If your stashed changes conflict with work done since the stash was created, you can apply the stash onto a new branch starting from the commit where the stash was originally made.

git stash branch new-feature-branch stash@{0}

Rewriting History

git rebase -i HEAD~n

Interactive rebase is the most powerful command for cleaning up commits before pushing. You can squash, reorder, reword, or drop commits.

git rebase -i HEAD~3

This opens your editor with the last 3 commits. Common operations:

squash: merge a commit into the previous one
reword: change the commit message
drop: remove the commit entirely

git commit --amend

Quick fix for the last commit, whether you forgot a file or made a typo in the message.

git add forgotten-file.ts
git commit --amend

Use --no-edit if you only want to add files without changing the message.

Debugging

git bisect

When a bug was introduced somewhere in the history and you have no idea where, bisect performs a binary search through your commits.

git bisect start
git bisect bad          # current commit is broken
git bisect good v1.0.0  # this tag was working
# git checks out a middle commit, you test it
git bisect good         # or git bisect bad
# repeat until git finds the first bad commit
git bisect reset

git blame -L start,end file

Shows who last modified each line. The -L flag lets you focus on a specific range instead of the entire file.

git blame -L 10,30 src/auth/login.ts

git log -S "search_term"

Finds commits that change the number of occurrences of a string. Use -G instead to match added or removed lines with a regular expression. See the git log documentation.

git log -S "validateToken" --oneline

Working with Remotes

git fetch --prune

Fetches updates from the remote and removes references to branches that no longer exist on the remote.

git fetch --prune

git push -u origin branch-name

The -u flag sets the upstream tracking reference. After this, you can just use git push and git pull without specifying the remote and branch.

git push -u origin feature/new-auth

git pull --rebase

git pull fetches and then integrates changes using the selected strategy. With --rebase, it replays local commits on top of the upstream history. A merge-based pull only needs a merge commit when histories diverge; a fast-forward needs none. Defaults depend on version and configuration. See git pull.

Figure: Your local commits X and Y diverged from origin/main at B. A pull using --no-rebase joins the diverged lines with merge commit M. A rebase pull replays them after D as new commits X′ and Y′, so history stays one straight line. (interactive version in the post)

git pull --rebase

You can make this the default behavior:

git config --global pull.rebase true

Recovery

git reflog

reflog records local reference updates, including recent HEAD movements that git log may not show. It can help recover committed work after a reset or rebase while the entries and objects remain available. It is not a backup of uncommitted files, and entries expire.

git reflog
git checkout HEAD@{3}

git cherry-pick

Applies a specific commit from another branch onto your current branch. Useful when you need one particular fix without merging an entire branch.

git cherry-pick abc1234

git branch from a commit

You can create a new branch starting from any commit in the history. This is useful when you realize an older commit would be a better starting point for a fix or feature, or when you need to recover work from a specific point in time.

# create a branch from a specific commit hash
git branch hotfix/login abc1234

# create and switch to it in one step
git checkout -b hotfix/login abc1234

# same thing using the newer switch command
git switch -c hotfix/login abc1234

You can also use references like tags, HEAD~n, or branch names instead of a commit hash:

# branch from 3 commits ago
git switch -c experiment HEAD~3

# branch from a tag
git switch -c patch-v2 v2.0.0

git restore --source origin/branch file

Restores a file to the version that exists on the remote. Useful when you have made local changes to a file and want to discard them in favor of the remote state.

# restore a single file to the remote main version
git restore --source origin/main -- src/auth/login.ts

# restore multiple files
git restore --source origin/main -- src/auth/login.ts src/auth/utils.ts

Make sure to git fetch first so your remote references are up to date:

git fetch origin
git restore --source origin/main -- path/to/file

Worktrees

A worktree lets you check out multiple branches of the same repository at the same time, each in its own directory on disk. Instead of stashing or committing unfinished work to switch branches, you create a new worktree and work on both branches simultaneously. Worktrees share the object database and most references, while each has its own index and HEAD. Git normally prevents checking out the same branch in two worktrees at once.

This is especially useful when you need to review a pull request while working on a feature, reproduce a bug on main without losing your current state, or run tests on one branch while developing on another.

git worktree add

Creates a new worktree linked to your repository. You can specify an existing branch or create a new one.

# check out an existing branch in a new directory
git worktree add ../hotfix-auth hotfix/auth

# create a new branch and check it out in a new directory
git worktree add -b feature/new-api ../new-api

Each worktree is a regular directory with a working tree. You can cd into it, run builds, tests, and commits independently.

git worktree list

Shows all worktrees linked to the repository.

git worktree list

Output looks like:

/home/user/project         abc1234 [main]
/home/user/hotfix-auth     def5678 [hotfix/auth]
/home/user/new-api         abc9012 [feature/new-api]

git worktree remove

Removes a worktree when you are done with it. The branch is kept, only the directory is cleaned up.

git worktree remove ../hotfix-auth

If the worktree has uncommitted changes, git will refuse to remove it unless you pass --force.

git worktree prune

Cleans up stale worktree references, for example when a worktree directory was deleted manually instead of using git worktree remove.

git worktree prune

Useful Aliases

Here are the aliases I keep in my .gitconfig:

[alias]
  st = status -sb
  lg = log --oneline --graph --all --decorate
  co = checkout
  br = branch -vv
  unstage = reset HEAD --
  last = log -1 HEAD --stat

These save keystrokes and make the output more readable. The small investment of setting them up pays off every day.

WSGI vs ASGI Protocols

Tue, 28 Jan 2025 00:00:00 GMT

Python web applications require protocols to communicate between web servers and application code. WSGI (Web Server Gateway Interface) and ASGI (Asynchronous Server Gateway Interface) are two such protocols, each with distinct characteristics and use cases.

WSGI: Synchronous Web Applications

WSGI is the traditional protocol for Python web applications, introduced in PEP (Python Enhancement Proposal) 333 and updated for Python 3 in PEP 3333. Its application interface is synchronous; a server can still handle concurrent requests using threads, processes, or compatible worker implementations.

Core Concepts

Single callable function (application object)
Synchronous request-response cycle
Compatible with most traditional web frameworks

Example WSGI Application

# wsgi_app.py
def simple_wsgi_app(environ, start_response):
    status = '200 OK'
    headers = [('Content-type', 'text/plain')]
    start_response(status, headers)

    return [b"Hello from WSGI!"]

# Test the WSGI app
def mock_start_response(status, headers):
    print(f"Status: {status}")
    print(f"Headers: {headers}")

response = simple_wsgi_app({}, mock_start_response)
print(response[0].decode())

gunicorn --workers=2 --bind 127.0.0.1:8000 wsgi_app:simple_wsgi_app

ASGI: Modern Async Applications

ASGI is a separate asynchronous interface designed as a successor to WSGI. Its protocol specifications cover HTTP (Hypertext Transfer Protocol) and WebSockets; actual HTTP/2 support depends on the server. See the ASGI specification.

Key Features

Async-first design
WebSocket support
HTTP/2 compatibility
Better handling of long-lived connections

Example ASGI Application

# asgi_app.py
import asyncio

async def simple_asgi_app(scope, receive, send):
    assert scope['type'] == 'http'

    await send({
        'type': 'http.response.start',
        'status': 200,
        'headers': [
            [b'content-type', b'text/plain'],
        ],
    })

    await send({
        'type': 'http.response.body',
        'body': b"Hello from ASGI!",
    })

# Test the ASGI app
responses = []
async def mock_send(message):
    responses.append(message)

async def main():
    await simple_asgi_app({'type': 'http'}, None, mock_send)
    print(f"Status: {responses[0]['status']}")
    print(responses[1]['body'].decode())

if __name__ == '__main__':
    asyncio.run(main())

uvicorn asgi_app:simple_asgi_app --lifespan off

When to Use Each

WSGI:

Simple request-response applications
Legacy systems integration
Applications built around synchronous libraries

ASGI:

Real-time applications
WebSocket requirements
High-concurrency needs
I/O-bound applications

Performance Considerations

ASGI can overlap I/O (Input/Output) efficiently when the application uses nonblocking libraries.
Blocking calls inside an async handler block its event-loop thread unless offloaded.
CPU-bound work needs an appropriate execution strategy with either interface, such as worker processes. Benchmark the actual application and server settings.

The following timeline is an idealized example with two synchronous WSGI workers, nonblocking database calls for ASGI, and no scheduling overhead; it is not a benchmark of the interfaces.

Figure: Three requests arrive at once. Each needs 5 ms of Python, an 80 ms database call, then 5 ms more. With two WSGI workers, the third request waits for a free worker and finishes at 180 ms. One ASGI worker overlaps all three database waits and finishes at 100 ms. (interactive version in the post)

Réflexions 2024

Thu, 09 Jan 2025 00:00:00 GMT

La solution la plus simple est généralement la meilleure. C'est pourquoi je cherche presque toujours à simplifier les choses.

Cette post est résumé des idées exposé dans les livres héritage Leçons d'hommes puissants du Humberto Montesinos M. and Père riche, père pauvre de Robert. En plus, j'ai ajouté quelques réflexions personnelles. En suivant les droits d'auteur, je voudrais mentionner que la plus part de idées ne sont pas les miennes, mais celles des auteurs mentionnés.

FOCUS

Souvent, nous avons tendance de perdre le focus on se concentre sur la solution en laissant le problème de côté. Focus en ce que vous pouvez contrôler, ne vous inquiétez pas de ce que vous ne pouvez pas contrôler o de ce qui est externe à vous.

Foyer! et rappelez-vous toujours FOCUS (Follow One Course Until Successful).

Ma philosophie de vie

pourquoi pas? imagine-toi ce qu'on pourrait accomplir.

Des opportunités augmentent à mesure qu'elles sont saisies. - Sun Tzu

Le peur d'agir ce qui nous empêche de réaliser nos rêves. C'est important de réfléchir, mais le plus important c'est d'agir. L'action est la clé du succès. Prend des risques c'est partie de la vie. L'impossible met seulement un peu plus de temps à arriver, autant vous insistez pour essayer et essayer.

Admirez ceux qui essaient, même s'ils échouent. - Seneca

Il n'y a rien d'impossible pour celui qui essaie. - Alexandre le Grand

Impossible est un mot que l'on ne trouve que dans le dictionnaire des imbéciles. - Napoléon Bonaparte

Je veux mettre l'accent sur l'importance d'agir consciencieusement.

Realize chaque action comme si elle était la dernière de votre vie. Marcus Aurelius

Certainement, les plans de ceux qui sont diligents auront de bons résultats; D'un autre côté, tous ceux qui se précipitent finiront dans la pauvreté - Rey Salomon

Reconnaître les limites de votre connaissance est la première étape pour apprendre.

Savoir que l'on sait ce que l'on sait, et que l'on ne sait pas ce que l'on ne sait pas, voilà la vraie sagesse. - Confucius

Les sages sont plus puissants que les forts, et ceux qui recherchent la connaissance deviennent de plus en plus forts - Rey Salomon

Objectifs

Prenez des décisions tous les jours. - Robert Kiyosaki

Il m'a fallu 10 ans pour avoir un succès du jour au lendemain - Woody Allen

Se fixer des objectifs est une partie importante de la vie. Soyez ambitieux!. être ambitieux est une qualité qui vous permettra de réaliser vos rêves. Il ne s'agit pas de faire plus, mais de faire le plus important. Priorisez vos objectifs et concentrez-vous sur eux.

Trouve une raison qui soi plus grande que la réalité. Payez-vous d'abord. Robert Kiyosaki, Père riche, père pauvre

Une raison ou un but est ce qui vous pousse à agir. Rappelle toi toujours pourquoi tu fais ce que tu fais. Une raison ou un but est la combinaison des désirs et no désirs de l'individu.

Du riz pour vivre et des fleurs pour avoir quelque chose pour quoi vivre. Nourrissez votre âme avec quelque chose de beau, un but. - Confucius

Avoir de gros problèmes

Considérer des questions triviales et ridicules comme problématiques témoigne d'une mentalité très limitée. Ce qui détermine si c'est bon ou mauvais, c'est votre interprétation.

À retenir : les problèmes sont des opportunités. Qu'est-ce que je dis problèmes, ce n'est pas un problème, c'est un défi. Un défi doit être abordé une étape à la fois.

Être un homme de parole

Quand on dit oui, que ce soit oui, et quand on dit non, que ce soit non, car tout ce qui est dit en plus de cela vient du mal. - Jésus-Christ

Normes

Vous êtes no seulement responsable de ce que vous fait, mais aussi de ce que vous ne fait pas. - Lao-Tse

Amis des Processus

Que est-ce que je dois me demander? Comment puis-je améliorer ce processus?. Oú est-ce que tu crois je suis en train de faire une erreur?. Avec quip pensez-vous que je devrais parler?.

Concurrency and Multithreading

Mon, 04 Nov 2024 00:00:00 GMT

Concurrency is the ability of a system or program to make progress on multiple tasks over overlapping periods. Parallelism means executing work at the same instant, for example on multiple CPU (Central Processing Unit) cores. Concurrency can allow better resource use and improve the overall performance of a system or program. In a concurrent system, tasks or processes may appear to be executing simultaneously, even if they are actually being interleaved or executed in parallel by multiple threads, processes, or coroutines. The main goal of concurrency is to improve the responsiveness, throughput, and scalability of a system or program.

What is Concurrency?

Concurrency means that one task can begin before another finishes. Tasks can take turns on one thread, overlap I/O (Input/Output) waits, or run in parallel when the runtime and hardware allow it.

What is Multithreading?

Multithreading is a specific form of concurrency where a single process can have multiple threads executing concurrently. A thread is not a subprocess. Unlike processes, which don't share memory by default and communicate through explicit IPC (Inter-Process Communication) mechanisms (pipes, shared memory, sockets), threads share the same memory space and can interact with each other more efficiently.

Why is Concurrency Important?

Efficiency: Concurrency helps applications handle multiple requests at the same time, making it faster and more responsive.
Scalability: As your application grows, you'll need to handle increasing amounts of data and tasks. Concurrent designs can overlap waits; parallel workers can also distribute suitable work across multiple cores.
Resource Utilization: Overlapping waits can keep a CPU busy. Multiple cores and simultaneous multithreading are different hardware features; concurrency alone guarantees neither full utilization nor lower energy use.
User Experience: For applications that require real-time interaction, such as gaming or video streaming, concurrency is critical for smooth performance.

How Does Concurrency Work in Different Environments?

1. In a Single-Threaded Environment

Node.js normally runs JavaScript callbacks on one event-loop thread, while the OS (Operating System) and a worker pool handle many I/O operations. Nonblocking I/O is well suited to this model. Long-running JavaScript and synchronous I/O can block the loop. See Node.js's event-loop guide.

2. In a Multithreaded Environment

In multithreaded environments, you can create multiple threads to handle different tasks concurrently. For example:

One thread can be dedicated to accepting new connections (e.g., in a web server).
Another thread can process the request.
Yet another thread can handle writing the response.

Figure: Three tasks that each compute for 20 ms, wait 60 ms on I/O, then compute 10 ms more. One thread running them in turn takes 270 ms. An event loop fills the waits with other work and finishes in 130 ms. Three threads on three cores finish in 90 ms. (interactive version in the post)

Why is Multithreading Important?

Multithreading is particularly important when dealing with I/O-bound operations, such as network requests or file operations. A blocking call makes its thread wait, but independent I/O operations need not be sequential. Threads or asynchronous I/O allow those waits to overlap with other work.

How to Implement Concurrency and Multithreading?

1. In JavaScript/Node.js

Node.js handles concurrency natively through its event loop and non-blocking async I/O. The event loop allows Node.js to perform I/O operations without blocking the main thread, delegating them to the system kernel or a thread pool (libuv) when possible.

For example:

Use Promises and async/await patterns to handle multiple asynchronous operations concurrently.
Use Worker Threads for CPU-bound tasks that would otherwise block the event loop.
Use frameworks like Express or Koa for building web applications on top of Node.js's concurrency model.

2. In Python

Python supports multithreading and multiprocessing to achieve concurrency. However, in a GIL (Global Interpreter Lock)-enabled CPython interpreter, only one thread can execute Python bytecode at a time. This means multithreading is effective for I/O-bound tasks but offers limited performance gains for CPU-bound tasks.

For example:

Use the threading module for basic multithreading (best for I/O-bound tasks).
Use multiprocessing if you need to run CPU-bound tasks concurrently, bypassing the GIL by using separate processes.
Use asyncio with async/await patterns to handle multiple asynchronous operations concurrently.

Starting with Python 3.13, an experimental free-threaded mode (no-GIL) was introduced via the --disable-gil build flag (PEP (Python Enhancement Proposal) 703). In Python 3.14, this free-threaded build became officially supported while remaining optional (PEP 779). It permits parallel Python execution, but speedups depend on the workload and synchronization costs. Some extension modules can re-enable the GIL. Native extensions that release the GIL can also run in parallel in a GIL-enabled build. See Python's free-threading guide.

Practical Examples of Concurrency

Web Servers

Imagine you're running a web server that handles multiple requests simultaneously. Without concurrency, each request would have to wait for the previous one to complete, leading to slow response times and high latency.

With concurrency:

One thread accepts new connections.
Another thread processes the request (e.g., reads data from the request and sends a response).

File Uploads

If you're uploading files to a server, each upload can be handled by a different thread. This ensures that multiple uploads happen at the same time without waiting for one another.

Machine Learning Models

In machine learning applications, you might want to train multiple models simultaneously on different cores. Native numerical libraries may release the GIL and use their own threads; processes or accelerators are other options. Benchmark the workload, since competing thread pools can reduce performance.

Common Challenges in Concurrency

Race Conditions: When multiple threads access shared data concurrently and at least one modifies it, the outcome depends on the timing of execution, leading to unpredictable results.
Deadlocks: When two or more threads are each waiting for a resource held by the other, none of them can proceed.
Starvation: A thread is perpetually denied access to resources because other threads are continuously prioritized over it.
Context Switching: In multithreaded environments, switching between threads (context switching) introduces overhead at the OS level due to saving/restoring thread state and CPU cache invalidation.

Key Takeaways

Concurrency is about making progress on tasks over overlapping periods; parallelism executes work simultaneously.
Multithreading is a specific implementation of concurrency where a single process has multiple threads.
In JavaScript/Node.js, concurrency is handled natively through the event loop and async I/O.
In Python, you can use threading or multiprocessing for basic concurrency.

Resources to Get Started

JavaScript/Node.js
- Express.js Documentation
- Understanding Async/Await in Node.js
Python
- Python Threading Tutorial
- Asyncio
Concurrency and Multithreading
- Concurrency and Async Await
- The Hitchhiker's Guide to Python: Concurrency

Functional Programming

Mon, 04 Nov 2024 00:00:00 GMT

Functional programming (FP) is a style that models programs as transformations of data. Instead of mutating state step by step, we build small functions and combine them to express intent.

Python is multi-paradigm, so we can use FP ideas without writing purely functional code all the time. In practice, this gives us readable pipelines, testable logic, and fewer side effects.

Why Use FP Concepts

FP principles are useful because they improve predictability.

Pure functions are easier to test.
Immutable data reduces accidental bugs.
Composition helps build reusable logic.
Stateless functions are easier to parallelize.

Core Concepts

Pure Functions

A pure function returns the same output for the same input and does not change external state.

def celsius_to_fahrenheit(celsius: float) -> float:
  return celsius * 9 / 5 + 32

The result depends only on celsius, and the function has no observable side effects. Avoiding global writes alone does not guarantee determinism: reading a clock or mutable global state can also change a result.

Immutability

Immutability means creating new values instead of mutating existing ones.

def add_item(items: tuple[str, ...], item: str) -> tuple[str, ...]:
  return items + (item,)

Using immutable structures makes data flow easier to reason about.

Higher-Order Functions

Higher-order functions receive functions as arguments or return functions.

def apply_twice(fn, value):
  return fn(fn(value))


print(apply_twice(lambda x: x + 3, 10))  # 16

This pattern lets us abstract behavior, not just data.

Referential Transparency

If an expression can be replaced by its value without changing behavior, it is referentially transparent. This simplifies debugging and refactoring.

For example, replacing double(5) with 10 is always safe if double is pure.

Function Composition

Composition means combining small functions into larger ones.

def compose(f, g):
  return lambda x: f(g(x))


strip_and_lower = compose(str.lower, str.strip)
print(strip_and_lower("  Hello  "))  # hello

Useful FP Tools in Python

Python includes many features that support FP style.

map, filter, and reduce

from functools import reduce

numbers = [1, 2, 3, 4, 5]

doubled = list(map(lambda x: x * 2, numbers))
evens = list(filter(lambda x: x % 2 == 0, doubled))
total = reduce(lambda acc, x: acc + x, evens, 0)

print(doubled)  # [2, 4, 6, 8, 10]
print(evens)    # [2, 4, 6, 8, 10]
print(total)    # 30

List comprehensions are often more readable than map and filter, but both approaches are valid.

partial for Function Specialization

from functools import partial


def power(base: int, exponent: int) -> int:
  return base ** exponent


square = partial(power, exponent=2)
cube = partial(power, exponent=3)

print(square(5))  # 25
print(cube(3))    # 27

This is partial application: fixing some arguments to obtain a new callable. Currying instead transforms a function of several arguments into a sequence of functions taking one argument each. See functools.partial.

Practical Pipeline Example

Suppose we receive numeric strings and blank lines and want the sum of positive amounts. This example skips blanks; malformed nonblank strings raise ValueError.

from typing import Iterable


def parse_amounts(lines: Iterable[str]) -> list[float]:
  return [float(line) for line in lines if line.strip()]


def only_positive(values: Iterable[float]) -> list[float]:
  return [v for v in values if v > 0]


def total(values: Iterable[float]) -> float:
  return sum(values)


raw = ["100", "-40", "", "18.5", "-2", "7.5"]
result = total(only_positive(parse_amounts(raw)))
print(result)  # 126.0

Each step is small, focused, and easy to test in isolation.

Recursion and Python

Recursion is common in FP, but Python does not optimize tail recursion. For large inputs, iterative solutions are often safer.

Use recursion when it improves clarity, but keep Python's recursion limit in mind.

Common Pitfalls

Overusing lambdas and making code harder to read.
Forcing reduce when sum or a loop is clearer.
Mixing pure and impure logic in the same function.
Ignoring readability in the name of style purity.

FP is most valuable when it improves maintainability, not when it is applied as a strict rule.

Final Thoughts

Functional programming in Python is about disciplined data transformation. You can gradually adopt it by writing pure helper functions, limiting mutation, and composing operations into clear pipelines.

Used well, FP makes code easier to test, reason about, and evolve.

References

Python Great Features

Mon, 04 Nov 2024 00:00:00 GMT

Duck Typing, ABCs, and Protocols

If it looks like a duck, swims like a duck, and quacks like a duck, then it probably is a duck.

The idea behind duck typing is that you don't need to check the type of an object before using it. Instead you care about what methods and properties the object has (API (Application Programming Interface)). If it has the methods and properties you need, you can use it.

Abstract Base Classes (ABCs) and Protocols enable us to keep the flexibility of duck typing, but also have some kind of type checking.

Abstract Base Classes (ABCs)

ABCs provide a way to enforce a contract between classes.

from abc import ABC, abstractmethod

class Animal(ABC):
    def __init__(self, name):
        self.name = name

    @abstractmethod
    def speak(self):
        raise NotImplementedError("speak() must be implemented")

class Dog(Animal):
    def speak(self):
        return 'Woof!'

dog = Dog("Rex")
print(dog.speak())

Protocols

The term protocols is used for some types supporting structural subtyping.

A structural type defines a set of values not by their class, but by their properties (e.g. attributes, methods, dictionary key/value types)

Protocols are a way to define a set of methods that a class must implement in order to satisfy that protocol during static type checking. Python does not automatically enforce it at runtime. @runtime_checkable permits limited attribute-presence checks, not validation of method signatures.

from typing import Protocol

class Animal(Protocol):
    def speak(self) -> str: ...

class Dog:
    def speak(self) -> str:
        return 'Woof!'

def speak(pet: Animal) -> str:
    return pet.speak()

print(speak(Dog()))  # Passes static type check

Type Hints

Type hints are a way to add type information to your code. They are not enforced by Python, but they can be used by tools like MyPy to check your code for type errors.

def add(x: int, y: int) -> int:
    return x + y

print(add(3, 5))

I will use type hints in the examples I provide in this post.

Functional Features

Closures

Functions can read or mutate state through globals, arguments, or enclosing scopes. A closure retains access to bindings in an enclosing function after that function returns; this can preserve state between calls.

A closure is a function object that remembers values in enclosing scopes. An outer function can return such a function, which continues to refer to its enclosing bindings after the outer call finishes. It means variables in the enclosing scope are kept in memory even after the function has finished executing.

from collections.abc import Callable

def request_token() -> str:
    return "my-secret-token"

def get_token() -> Callable[[], str]:
    _token: str | None = None
    def _get():
        nonlocal _token
        if _token:
            print('Using cached token')
            return _token

        print('Fetching new token')
        # call some API to get the token
        _token = request_token()
        return _token
    return _get

token = get_token()
print(token())
print(token())
print(token())

Fetching new token
my-secret-token
Using cached token
my-secret-token
Using cached token
my-secret-token

The next example uses a mock HTTP (Hypertext Transfer Protocol) client to illustrate token caching. A real client also needs token expiry handling, failed-response handling, and synchronization if concurrent callers may refresh the same token. The examples with top-level await run in an async-capable notebook or the page runner; in a regular script, put the async work inside main() and use asyncio.run(main()).

import asyncio
import base64
from collections.abc import (
    Callable,
    Awaitable,
)
import os

# Mock for browser execution (aiohttp requires server environment)
class _MockResponse:
    async def json(self):
        return {"token": "mock-jwt-token"}
    async def __aenter__(self):
        return self
    async def __aexit__(self, *a):
        pass

class ClientSession:
    def __init__(self, base_url=""):
        pass
    def post(self, url, **kw):
        return _MockResponse()
    async def __aenter__(self):
        return self
    async def __aexit__(self, *a):
        pass

COMMON_HEADERS = {"Content-Type": "application/json"}
os.environ["PUBLIC_KEY"] = "test_public_key"
os.environ["SECRET_KEY"] = "test_secret_key"


def _get_access_token() -> Callable[[ClientSession], Awaitable[str]]:
    current_token: str | None = None

    async def _access_token(session: ClientSession) -> str:
        nonlocal current_token

        if current_token:
            print('Using cached token')
            return current_token

        print('Fetching new token')
        PUBLIC_KEY = os.getenv("PUBLIC_KEY")
        SECRET_KEY = os.getenv("SECRET_KEY")
        text = f"{PUBLIC_KEY}:{SECRET_KEY}"
        encode = base64.b64encode(text.encode("utf-8"))
        token = str(encode, "utf-8")
        authorization = f"Basic {token}"
        headers = {**COMMON_HEADERS, "Authorization": authorization}
        async with session.post(
            "/login",
            headers=headers,
        ) as response:
            content = await response.json()
            if content and content.get("token"):
                current_token = content["token"]

        if current_token is None:
            raise ValueError("Token not found")

        return current_token

    return _access_token


_get_access_token = _get_access_token()

async def main() -> None:
    async with ClientSession("https://api.example.com/") as session:
        token = await _get_access_token(session)
        token = await _get_access_token(session)
        token = await _get_access_token(session)
        token = await _get_access_token(session)

await main()

Fetching new token
Using cached token
Using cached token
Using cached token

Decorators (Higher order functions)

Decorators are a way to wrap a function and add extra functionalities to such a function.

In the following example we define a decorator that adds an access token to the headers of a request. Note that the decorator receives a function that receives a ClientSession instance as first argument, that instance will be used to get the access token without having to open a new session for each request.

from collections.abc import (
    Awaitable,
)
from functools import wraps
from typing import (
    Callable,
    Concatenate,
    ParamSpec,
    TypeVar,
)

# Mock for browser execution (aiohttp requires server environment)
class _MockResponse:
    def __init__(self, data):
        self._data = data
    async def json(self):
        return self._data
    async def __aenter__(self):
        return self
    async def __aexit__(self, *a):
        pass

class ClientSession:
    def __init__(self, base_url=""):
        pass
    def get(self, url, **kw):
        return _MockResponse({"id": 1, "name": "Alice"})
    async def __aenter__(self):
        return self
    async def __aexit__(self, *a):
        pass

async def _get_access_token(session):
    return "mock-token"

T = TypeVar('T')
P = ParamSpec('P')


def add_headers(
    func: Callable[Concatenate[ClientSession, P], Awaitable[T]],
) -> Callable[Concatenate[ClientSession, P], Awaitable[T]]:
    @wraps(func)
    async def decorated(
        session: ClientSession, /, *args: P.args, **kwargs: P.kwargs
    ) -> T:
        token = await _get_access_token(session)
        headers = {
            "Content-Type": "application/json",
            "Accept": "application/json",
            "Authorization": f"Bearer {token}",
        }
        kwargs["headers"] = headers

        return await func(session, *args, **kwargs)

    return decorated


@add_headers
async def get_user(
    session: ClientSession,
    user_id: int,
    *,
    headers: dict[str, str] | None = None
) -> dict[str, object]:
    async with session.get(f"users/{user_id}", headers=headers) as resp:
        return await resp.json()

async with ClientSession("https://api.example.com/") as session:
    user = await get_user(session, 1) # headers will be added by the decorator
    print(user)

wraps

without @wraps the name of the function will be wrapper_function

def mock_decorator(original_function):
    def wrapper_function(*args, **kwargs):
        return original_function(*args, **kwargs)
    return wrapper_function

@mock_decorator
def f(x):
    '''Docstring'''
    return x ** 2

print(f.__name__)
print(f.__doc__)

wrapper_function
None

from functools import wraps

def mock_decorator(original_function):
    @wraps(original_function)
    def wrapper_function(*args, **kwargs):
        return original_function(*args, **kwargs)
    return wrapper_function

@mock_decorator
def f(x):
    '''Docstring'''
    return x ** 2

print(f.__name__)
print(f.__doc__)

f
Docstring

def stateful_function(func):
    cache = {}
    def wrapper_function(*args, **kwargs):
        key = (args, tuple(sorted(kwargs.items())))
        if key not in cache:
            cache[key] = func(*args, **kwargs)
        return cache[key]
    return wrapper_function

@stateful_function
def fibonacci(n):
    if n < 2:
        return n
    return fibonacci(n-1) + fibonacci(n-2)

print(fibonacci(10))

The manual cache above requires hashable arguments and keyword values; it is only a teaching example. Stringifying arguments is not a reliable cache key, because different objects can have the same string representation. For pure functions, prefer functools.cache or lru_cache.

cache

import time
from functools import cache

def timer(function):
    def wrapper(*args, **kwargs):
        start = time.time()
        result = function(*args, **kwargs)
        end = time.time()
        print(f'Elapsed time: {end - start}')
        return result
    return wrapper

@cache
def fibonacci(n):
    if n < 2:
        return n
    return fibonacci(n-1) + fibonacci(n-2)

@timer
def fibonacci_timer(n):
    return fibonacci(n)

print(fibonacci_timer(10))

This is kinda like a cache (memoization), so that the function doesn't have to be called again if the arguments are the same.

Working with Files

Generators

Generators are a special class of functions that simplify the task of writing iterators. Generators are a simple and powerful tool for creating iterators. They are written like regular functions but use the yield statement whenever they want to return data. Each time next() is called on it, the generator resumes where it left off (it remembers all the data values and which statement was last executed).

Generators are iterators, a kind of iterable you can only iterate over once. It’s because they do not store all the values in memory, they generate the values on the fly, that allows processing of large amounts of data efficiently.

# Create a sample file for demonstration
with open('file.txt', 'w') as f:
    f.write('Hello from a large file!\n' * 10)

def read_large_file(filename):
    with open(filename) as file:
        while True:
            chunk = file.read(1000)
            if not chunk:
                break
            yield chunk

for chunk in read_large_file('file.txt'):
    print(chunk)

def fibonacci(n):
    a, b = 0, 1
    for _ in range(n):
        yield a
        a, b = b, a + b

for i in fibonacci(10):
    print(i)

Context Managers

Context managers are used to allocate and release resources precisely when you want to. They are very useful when you are working with external resources like files, network connections, etc. Context managers are normally implemented using a class that implements the special methods __enter__() and __exit__(). The __enter__() method is invoked when the with statement is encountered. The __exit__() method is invoked at the end of the with block.

class OpenFile:
    def __init__(self, filename, mode):
        self.filename = filename
        self.mode = mode

    def __enter__(self):
        self.file = open(self.filename, self.mode)
        return self.file

    def __exit__(self, exc_type, exc_val, traceback):
        self.file.close()

with OpenFile('sample.txt', 'w') as f:
    f.write('Testing')

with open('sample.txt', 'r') as f:
    print(f.read())

import time

class Timer:
    def __init__(self):
        self.start = None
        self.end = None

    def __enter__(self):
        self.start = time.time()
        return self

    def __exit__(self, exc_type, exc_val, traceback):
        self.end = time.time()

    def elapsed_time(self):
        return self.end - self.start

with Timer() as timer:
    print('This should take approximately 2 seconds')
    time.sleep(2)

print('Elapsed time: {}'.format(timer.elapsed_time()))

Note that the __exit__() method can optionally return a Boolean value. If it returns True, any exception raised within the with block is suppressed and execution proceeds as if no exception had occurred. If it returns False, any exception raised within the with block propagates to the surrounding code. A return value of None also leaves the exception unsuppressed.

import threading

lock = threading.Lock()

class LockedContext:
    def __init__(self, lock):
        self.lock = lock

    def __enter__(self):
        print('acquiring lock')
        self.lock.acquire()

    def __exit__(self, exc_type, exc_val, traceback):
        print('releasing lock')
        self.lock.release()

with LockedContext(lock):
    print('Lock acquired')

# The lock is automatically released when the with block ends,
# even if an error occurs

Python Virtual Environments

Mon, 04 Nov 2024 00:00:00 GMT

Virtual environments are used to create an isolated environment for Python projects. This means that each project can have its own dependencies, regardless of what dependencies every other project has. First of all let's review how to install, update, and uninstall packages in a virtual environment with pip. Note: The shell examples below target Linux and macOS. Create and activate an environment first (python3 -m venv .venv, then source .venv/bin/activate); python -m pip then targets that environment. Braced names are placeholders.

Contains a specific Python interpreter and software libraries and binaries which are needed to support a project (library or application). These are by default isolated from software in other virtual environments and Python interpreters and libraries installed in the operating system. Python docs

Freezing packages (PIP)

To freeze the packages installed in the virtual environment, you can use the freeze command. This will create a requirements.txt file with all the installed distributions and their versions (some packaging tools are omitted by default). This is an environment snapshot, not a dependency resolver or a cross-platform lockfile.

python -m pip freeze > requirements.txt

Installing packages (PIP)

python -m pip install {package_name}
python -m pip install -r requirements.txt

Uninstalling packages

python -m pip uninstall {package_name}

Updating packages

python -m pip install --upgrade {package_name}

Creating a virtual environment

To create a virtual environment, you could use the venv module that comes with Python, this is the recommended way to create virtual environments.

venv (built-in module in python 3.3+)

Extracted from python venv docs A virtual environment is (amongst other things):

Used to contain a specific Python interpreter and software libraries and binaries which are needed to support a project (library or application). These are by default isolated from software in other virtual environments and Python interpreters and libraries installed in the operating system.

Contained in a directory, conventionally named .venv or venv in the project directory, or under a container directory for lots of virtual environments, such as ~/.virtualenvs.

Not checked into source control systems such as Git.

Considered as disposable – it should be simple to delete and recreate it from scratch. You don't place any project code in the environment.

Not considered as movable or copyable – you just recreate the same environment in the target location.

# Run the venv module as a script with the -m option, passing the route
# where the virtual environment info will be store (environment name).
python3 -m venv {env_name}
source {env_name}/bin/activate
python -m pip install -r requirements.txt
python -m pip freeze > requirements.txt
deactivate

pipenv

Extracted from pipenv docs The problems that Pipenv seeks to solve are multi-faceted:

You no longer need to use pip and virtualenv separately: they work together.
Managing a requirements.txt file with package hashes can be problematic. Pipenv uses Pipfile and Pipfile.lock to separate abstract dependency declarations from the last tested combination.
Hashes are documented in the lock file which are verified during install. Security considerations are put first.
Strongly encourage the use of the latest versions of dependencies to minimize security risks arising from outdated components.
Gives you insight into your dependency graph (e.g. $ pipenv graph).
Streamline development workflow by supporting local customizations with .env files.

python -m pip install pipenv
cd {project_dir}
pipenv install -r requirements.txt
pipenv uninstall {package_name}
pipenv install -e git+repo_url@branch#egg=package_name
# Run a script using pipenv run
pipenv run python main.py
# install a package from a git repo
pipenv shell
pipenv lock # creates a Pipfile.lock file
exit

Install all the packages in the Pipfile.lock file (replicates the environment)

pipenv install --ignore-pipfile
pipenv graph # shows the dependency graph
pipenv check --scan # vulnerability scan; may require service credentials
exit

virtualenv (Lower level)

virtualenv can help you. It creates an environment that has its own installation directories, that doesn't share libraries with other virtualenv environments (and optionally doesn't access the globally installed libraries either). virtualenv docs

# basic usage
python -m pip install virtualenv
cd project_folder
virtualenv venv
source venv/bin/activate
pip install requests
deactivate

# specify the python version, note that the python version must be installed
# in the system.
virtualenv --python=python3.14 {env_name}
# lsvirtualenv belongs to the separate virtualenvwrapper package
source {env_name}/bin/activate
pip install -r requirements.txt # All other pip commands work as usual
deactivate
rm -rf {env_name} # remove the virtual environment

To delete a virtual environment, just delete its folder. (In this case, it would be rm -rf venv.)

Isolating the versions of Python

Different projects may require different versions of Python. To isolate the versions of Python, you can use pyenv or pyenv-virtualenv.

pyenv

pyenv installs and selects Python interpreters; it does not create virtual environments by itself. Install it using the official instructions, including shell initialization and build dependencies. It is not installed with pip. Upstream pyenv targets Unix-like systems; Windows users can use WSL (Windows Subsystem for Linux) or the separate pyenv-win project.

pyenv install --list
pyenv install {version}
pyenv versions
pyenv local {version}
python -m venv .venv
source .venv/bin/activate

pyenv global sets the user-wide default, while pyenv local writes the project's .python-version.

pyenv-virtualenv

The optional pyenv-virtualenv plugin adds pyenv virtualenv, pyenv virtualenvs, and activation commands. Install and initialize that plugin before using these commands.

pyenv install {version}
pyenv virtualenv {version} {env_name}
pyenv local {env_name}

Anaconda

Anaconda Distribution bundles Python and scientific-computing packages. Conda is an open-source environment and package manager, but Anaconda Distribution and its hosted repositories have separate terms; free use depends on eligibility. Check the Anaconda terms. Install the current distribution for your operating system from the official site before running the following Conda examples.

conda info
conda create --name {env_name} python={version}
conda activate {env_name}
conda install {package_name}
conda list
conda update {package_name}
conda remove {package_name}
conda deactivate
conda remove --name {env_name} --all
conda create --name {new_env} --clone {source_env} # clone an environment
conda env export > environment.yml # export an environment
conda env create -f environment.yml # create an environment from a yml file
conda env list
conda env remove -n {env_name} # remove an environment

some useful commands

conda list # list all the packages installed in the current environment
# list all the packages installed in the current environment with their versions
conda list --explicit
# install a package from a specific channel
conda install --channel conda-forge {package_name}
conda list --revisions # list all the revisions of the environment
 # install a specific revision of the environment
conda install --revision {revision_number}
# export the environment to a yml file
conda env export --from-history --file environment.yml

Mamba

Mamba is a fast, drop-in replacement for the conda package manager. It is a C++ implementation of the conda package manager, that uses multi-threading and a sophisticated dependency solver to achieve maximum performance.

conda install -c conda-forge mamba
mamba --help
mamba create --name {env_name} python={version}
mamba activate {env_name}
mamba install {package_name}
mamba list
mamba update {package_name}
mamba remove {package_name}
mamba deactivate
mamba env create -f environment.yml # create an environment from a yml file

Docker

Docker is a containerization platform that allows you to create and run containers. Docker containers are isolated environments that run on a single host. To install docker on ubuntu, follow this tutorial.

Containerizing a Python script

# Dockerfile
FROM python:3.14-slim
WORKDIR /app
COPY ["requirements.txt", "/app"]
RUN python -m pip install --no-cache-dir -r requirements.txt
COPY app.py /app/app.py
CMD ["python", "app.py"]

Alternatively, you can use docker compose to build and run the docker container.

# compose.yaml
services:
  app-csv:
    build:
      context: .
      dockerfile: Dockerfile
    volumes:
      - .:/app

docker compose build
docker compose up -d
docker compose ps
docker compose logs
docker compose exec app-csv bash
cd /app
python3 app.py
exit
docker compose down

Containerizing an ASGI app

This assumes app.py exports an ASGI (Asynchronous Server Gateway Interface) application named app (for example, FastAPI) and requirements.txt includes Uvicorn and the framework. A Flask app uses WSGI (Web Server Gateway Interface) and needs a WSGI server or an ASGI adapter. The Dockerfile syntax is documented in the Dockerfile reference.

# Dockerfile
FROM python:3.14-slim
WORKDIR /app
COPY ["requirements.txt", "/app"]
RUN python -m pip install --no-cache-dir -r requirements.txt
COPY . /app
CMD ["uvicorn", "app:app", "--host", "0.0.0.0", "--port", "80"]

# compose.yaml
services:
  app-asgi:
    build:
      context: .
      dockerfile: Dockerfile
    volumes:
      - .:/app
    ports:
      - "80:80"

docker compose build
docker compose up -d
docker compose ps

Choosing a tool

Use venv and pip for a simple Python-only project.
Use Pipenv when you want a Pipfile and lockfile workflow.
Use Conda or Mamba when managing Python together with native dependencies.
Use pyenv when you need to install and select multiple Python interpreters.

These are alternative workflows; you do not need to stack all of them.

Divide and conquer

more info in the Snakemake documentation

Terminal Basics

Mon, 04 Nov 2024 00:00:00 GMT

The examples target Bash on GNU (GNU's Not Unix)/Linux unless stated otherwise. macOS uses BSD (Berkeley Software Distribution) versions of several utilities, and flags can differ. Replace angle-bracketed placeholders before running commands; quote paths containing spaces.

What is a command?

A command is a program that you can run from the terminal, it is also a utility of the shell, a shell function, or an alias. A command can receive arguments and options.

What is a shell?

A shell is a command line interpreter, it is a program that reads commands from the keyboard and executes them.

Bourne shell (sh)
Bash shell (bash)
C shell (csh)
Korn shell (ksh)
Z shell (zsh)
Fish shell (fish)
PowerShell (powershell)

What is a terminal?

A terminal emulator provides a text interface to a shell or another program. It often runs inside a graphical desktop window; the shell is the command interpreter running within it.

Commands

ls

ls -al : list files in long format, including hidden files
ls -lrt : list files in long format, sorted by modification time, oldest first
ls -lh : list files in long format, with human readable sizes

tree : list files in a tree format

tree -L 2 : list files in a tree format, 2 levels deep.

cd [path]

cd : move to home directory
cd -: change directory to previous directory
cd .. : change directory to parent directory
cd / : change directory to root directory
cd <path>: change directory to the specified path
cd ~<user>: change directory to user's home directory
cd ~: change directory to current user's home directory

pwd (print working directory)

pwd

managing files

touch

touch file_name: create an empty file if absent; otherwise update its timestamps.
touch file_1_name file_2_name : create file1 and file2
> file_name : create an empty file or truncate an existing file to zero bytes

cp

cp file_name file_name_copy: copy file_name to file_name_copy.
cp -r dir_name dir_name_copy: copy dir_name to dir_name_copy.
cp -p : copy files preserving attributes
cp -a : copy files preserving attributes and links
cp -i : prompt before overwriting existing files
cp -f : force overwrite existing files

mv

mv <file_name> <path> : move file to specified path
mv <file1> <file2> : rename file1 to file2
mv -i : prompt before overwriting existing files
mv -f : force overwrite existing files
mv <dir1> <path>: move dir1 to specified path
To include hidden entries in Bash, use dotglob and nullglob, then pass the expanded array to mv. The destination must be outside the source:
```
(
  shopt -s dotglob nullglob
  files=(source_dir/*)
  if ((${#files[@]})); then
    mv -- "${files[@]}" destination_dir/
  fi
)
```

mkdir

mkdir <dir_name>: create dir_name directory.
mkdir <dir1> <dir2> <dir3> : create directories dir1, dir2, dir3
mkdir -p : create parent directories as needed

cat head tail less

cat <file_name>: print file_name contents.
cat > <file_name> : create file and write to it.
cat <file_name1> <file_name2> : print file1 and file2 contents.
head <file_name>: print the first 10 lines of file_name.
head -n 20 <file_name>: print the first 20 lines of file_name.
tail <file_name>: print the last 10 lines of file_name.
tail -n 20 <file_name>: print the last 20 lines of file_name.
less <file_name>: print file_name contents in less.
less +20 <file_name>: print file_name contents in less starting at line 20.
less +/text <file_name>: print file_name contents in less starting at the first occurrence of text.
less -N <file_name>: print file_name contents in less with line numbers.
less -S <file_name>: print file_name contents in less without wrapping lines.
less -F <file_name>: exit automatically if the contents fit on one screen.
less -R <file_name>: display supported color escape sequences.
less -X <file_name>: skip terminal initialization/deinitialization sequences.
less -Q <file_name>: suppress the terminal bell.
less -K <file_name>: exit on an interrupt, such as Ctrl-C.
less -M <file_name>: use a more detailed prompt.
less -i <file_name>: case-insensitive search unless the pattern has uppercase letters.
less -I <file_name>: always ignore case in searches.
less -I -N -S <file_name>: combine case-insensitive search, line numbers and unwrapped lines.

less

less file_name: print file contents, one page at a time.
- /word_to_search_for : allow you to search for a word in the file.
- q : quit.

file

file <path> : print file type

rm

rm <file_name>: delete file_name.
rm -i <file> : prompt before removing a file
rm -f <file> : remove without prompting; ignore missing files
rm -r <dir> : remove directories recursively
rmdir <dir_name>: remove an empty directory.
rm -rf <dir_name>: delete dir_name and its contents.

xdg-open / open

xdg-open file_name: open with the default desktop application on Linux.
open file_name: use the default application on macOS.

nautilus

nautilus [path_to_folder] : open file manager

type

type command: print command type

alias

alias: list aliases
alias foo="bar": create alias foo for bar

help

In Bash, help is a built-in that documents Bash built-ins; it does not provide the manual for arbitrary external programs.

help command: print command help

man

user manual for commands

man command: print command manual

info

info command: print command info

whatis

whatis command: print command description

wildcards

Allows us to match patterns in filenames.

*: matches any number of characters.
ls *.txt: matches any file with .txt extension.
ls data*.txt: list files with .txt extension starting with data.
?: matches any single character.
??: matches any two characters and so on.
ls data?.txt: list files with .txt extension starting with data and a single character.
[abc]: matches a, b, or c.
[!abc]: matches anything but a, b, or c.
ls [[:upper:]]*: list matching uppercase-leading names; for matching directories, ls lists their contents unless -d is used.
ls -d [[:upper:]]*/: list matching directories starting with an uppercase letter.
ls -d [[:lower:]]*/: list matching directories starting with a lowercase letter.
ls [ad]*: list files starting with a or d followed by any number of characters.
ls -l [2-9]*: list files starting with a number between 2 and 9.
ls -l [0-9][0-9]*: list files starting with two numbers.

redirection

A redirection is a way to redirect the output of a command to a file or another command. file descriptor (0-1-2).

standard input (stdin) : 0
standard output (stdout) : 1
standard error (stderr) : 2
command < file_name: read from file.
command > file_name: write stdout of the specified command to file.
command >> file.txt : append stdout to file.
ls 2> file_name: write stderr to file.
ls 2>> file_name: append standard error to file.
ls > file_name 2>&1: write standard output and standard error to file.

pipes

A pipe is a way to connect the output of one command to the input of another command.

echo "foo" | grep "foo": print "foo" and grep for "foo".
cat file1 file2 | grep "foo": print file1 and file2 and grep for "foo".
ls -lh | tee file_name | less: list files in long format with human readable sizes, write to file, and print one page at a time.
ls -lh folder | sort | tee file_name | less: list files in long format with human readable sizes, sort, write to file, and print one page at a time.
cowsay "foo" | lolcat: print "foo" with cowsay and lolcat.
echo "foo" | cowsay | lolcat: print "foo" with cowsay and lolcat.

control operators

Control operators determine sequencing and conditions. ;, &&, and || do not by themselves create subshells; & starts an asynchronous job in Bash.

ls; echo "foo": list files and print "foo".
ls; mkdir dir; cal: list files, create directory, and print calendar. (asynchronous "&" creates a background process).
ls & echo "foo": run ls in the background and echo in the foreground.
ls && echo "foo": list files and print "foo" if ls succeeds.
ls || echo "foo": list files and print "foo" if ls fails.

permissions

A permission is a way to control access to a file or directory.

ls -l begins with a type character: - for a regular file, d for a directory, l for a symbolic link, and b for a block device. A block device is a device that is accessed in fixed-size blocks.

Kind of permissions

r : read
w : write
x : execute

owner	group	other
rwx	r-x	r-x
111	101	101
7	5	5

Symbolic mode

u : user g : group o : other a : all chmod u=rwx,g=rwx,o=rwx file : set permissions to 777

root user: a root user has full access to the system.
chmod : change file permissions (change mode)(octal).
chmod 755 file_name: set permissions to 755. (owner: rwx, group: rx, other: rx).
chmod u=rwx,g=rwx,o=rwx file_name: set permissions to 777. (owner: rwx, group: rwx, other: rwx).
chmod u-rwx,g-rwx,o-rwx file_name: set permissions to 000. (owner: ---, group: ---, other: ---).
chmod g+w file_name: add group write permission, preserving all other bits.
whoami: print current user.
id: print current user and group.
su root: switch to root user.
su user: switch to user.
in ubuntu: sudo su :switch to root user.
passwd: change password.
chown -R "$(id -un):$(id -gn)" folder_name: recursively set owner and primary group to the current user; sufficient privileges are required.

change password

passwd : change password

managing permissions

chmod u=rwx,g=rwx,o=rwx file_name: set permissions to 777
chmod u=rwx,g=rwx,o=rx file_name: set permissions to 775
chmod u=rwx,g=rwx,o= file_name: set permissions to 770
chmod u=rwx,g=rwx file_name: set user and group to rwx; leave other bits unchanged
chmod u=rwx,go=w file_name: set ordinary permission bits to 722
chmod 777 file_name: change file_name permissions to 777.
chmod 755 file_name: change file_name permissions to 755.
chmod 644 file_name: change file_name permissions to 644.
chmod 600 file_name: change file_name permissions to 600.
chmod 400 file_name: change file_name permissions to 400.
chmod 000 file_name: change file_name permissions to 000.
chmod u+x file_name: add execute permission to file_name for user.
chmod u-x file_name: remove execute permission to file_name for user.
chmod u+r file_name: add read permission to file_name for user.
chmod u-r file_name: remove read permission to file_name for user.
chmod u+w file_name: add write permission to file_name for user.
chmod u-w file_name: remove write permission to file_name for user.
chmod u+x,g+x,o+x file_name: add execute permission to file_name for user, group and others.
chmod u-x,g-x,o-x file_name: remove execute permission to file_name for user, group and others.
chmod u+r,g+r,o+r file_name: add read permission to file_name for user, group and others.
chmod u-r,g-r,o-r file_name: remove read permission to file_name for user, group and others.
chmod u+w,g+w,o+w file_name: add write permission to file_name for user, group and others.
chmod u-w,g-w,o-w file_name: remove write permission to file_name for user, group and others.
chmod u+x,g+x,o+x,u-r,g-r,o-r,u-w,g-w,o-w file_name: add execute permission to file_name for user, group and others and remove read and write permission to file_name for user, group and others.

environment variables

Environment variables are name/value pairs inherited by child processes. In Bash, assignment creates a shell variable; export makes it part of the environment passed to subsequent commands.

ln -s /home/user/dir link_name: create symbolic link to dir named link_name.
printenv: print environment variables
echo $VARIABLE_NAME: print VARIABLE_NAME environment variable
echo $PATH: /usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin: /sbin:/bin:/usr/games:/usr/local/games:/snap/bin --> the directories where the shell looks for commands
echo $HOME: print HOME environment variable -- /home/user -- the home directory of the current user
echo $USER : print USER environment variable -- user -- the current user
echo $SHELL : print SHELL environment variable -- /bin/bash -- the shell program

how to modify environment variables?

export PATH=$PATH:/home/user/bin: add /home/user/bin to PATH environment variable

look for the file .bashrc (bash) in the home directory of the user.
open the file .bashrc with a text editor to who is using bash. In mac .zshrc. That's could vary depending on the shell you are using.
add the line export PATH=$PATH:/home/user/bin to the file .bashrc.
to add a environment variable, add the line export VARIABLE_NAME='value' to the file .bashrc.
to add an alias, add the line alias alias_name='command' to the file .bashrc.
to add bins to the path, add the line export PATH=$PATH:/home/user/bin to the file .bashrc.
save the file .bashrc.

Searching command

A search command is a way to search for a command.

which <command> : print command path. If the command is not found, nothing is printed.
which code: /usr/bin/code --> the path to the command code
find path -name "file_name": search for file_name in path. segment by name (wildcards and find):
find path -name "*.txt": search for files with .txt extension in current directory. segment by type:
find path -type f -name "*.txt" : search for files with .txt extension in the specified path.
find ./ -type f -name "*.log": search for files with .log extension in current directory.
find ./ -type d -name dir: search for directories with name dir in current directory.
find ./ -type d -empty: search for empty directories in current directory.
find ./ -maxdepth 2 -type d: search for directories in current directory with a maximum depth of 2
find ./ -mindepth 2 -type d: search for directories in current directory with a minimum depth of 2 segment by size:
find ./ -type f -size +100k: search for files in current directory with a size greater than 100k.
find ./ -size +100M: search for files larger than 100M in current directory.
find ./ | less: search for files in current directory and print the results in less. segment by date:
find ./ -type f -mtime +1: search for files in current directory that were modified at least two full 24-hour periods ago (-mtime rounds down). segment by permission:
find ./ -type f -perm 777: search for files in current directory with permissions 777.
find ./ -type f -perm -777: search for files in current directory with all of the listed permission bits set (additional special bits may be set).
find ./ -type f -perm /777: search for files in current directory with at least one listed permission bit set (GNU find syntax). segment by owner:
find ./ -type f -user user: search for files in current directory with owner user.
find ./ -type f -group group: search for files in current directory with group group. segment by content:
find ./ -type f -exec grep -l 'text' {} \;: search for files in current directory with the text text and print the file name.
find ./ -type f -exec grep -q 'text' {} \; -ok rm {} \;: find matching files and ask before deleting each; this preserves filenames containing spaces.
find ./ -type f -size +100k -exec rm {} \;: search for files in current directory with a size greater than 100k and delete them.

managing directories

A directory is a way to manage directories.

pwd: print working directory.
cd dir_name: change directory to dir_name.
cd ..: change directory to parent directory.
cd ~: change directory to home directory.
cd -: change directory to previous directory.
ls: list directory contents.
ls -a: list directory contents including hidden files.
ls -l: list directory contents in long format.
ls -la: list directory contents in long format including hidden files.
ls -lh: list directory contents in long format with human readable sizes.
ls -lah: list directory contents in long format with human readable sizes including hidden files.
ls -lt: list directory contents in long format sorted by modification time.
ls -lat: list directory contents in long format sorted by modification time including hidden files.
ls -lS: list directory contents in long format sorted by size.
ls -lSr: list directory contents in long format sorted by size in reverse.
ls -lX: list directory contents in long format sorted by extension.
ls -lXr: list directory contents in long format sorted by extension in reverse.
GNU ls -lU: use directory order, without sorting. On macOS, -U has different timestamp semantics; check the local man ls.
ls -lF: list directory contents in long format with a slash after directories.

whereis

whereis locates binaries, manual pages and source locations for command names; it is not a general file search.

whereis <file_name>: find file_name.
whereis -b <file_name>: find file_name binaries.
whereis -m <file_name>: find file_name manuals.
whereis -s <file_name>: find file_name sources.

ln

ln -s : create symbolic links

find

Quote patterns so find, rather than the shell, interprets them:

find . -type f -name '*.txt'
find . -type f -name '*.txt' -exec grep -n 'foo' {} +

The first command finds text files recursively. The second searches their contents and safely handles filenames containing spaces. Permission predicates are bit tests, not numeric comparisons; see the GNU find mode-bit reference.

grep

A grep is a way to search for a pattern in a file.

grep <regex> <file_name>: search for regex in file_name.
grep -r <regex> <file_name>: search for regex in file_name recursively.
grep -i <regex> <file_name>: search for "foo" in files, case insensitive.
grep -v <regex> <file_name> : invert match, print non-matching lines.
grep -l <regex> <file_name>: print names of files with at least one matching line
grep -c <regex> <file_name>: count matching lines, not individual occurrences.
grep -n <regex> <file_name>: print line numbers.
grep -A 2 <regex> <file_name>: search for regex in files, print 2 lines after match
grep -B 2 <regex> <file_name>: search for regex in files, print 2 lines before match
grep -C 2 <regex> <file_name>: search for regex in files, print 2 lines before and after match
wc <file_name>: count newline characters, words and bytes.
wc -l <file_name>: count lines in file_name.
wc -w <file_name>: count words in file_name.
wc -c <file_name>: count bytes. Use wc -m to count characters.

Network utilities

These commands inspect interfaces or communicate with network services.

ifconfig: print network interfaces.
ping <host-or-ip>: send ICMP (Internet Control Message Protocol) echo requests; a missing reply can reflect filtering
curl <url>: print url contents.
curl <url> > index.html: write url contents (stdout) to index.html.
wget <url>: download url. get form url.
traceroute <host-or-ip>: probe hops toward a destination; some may not reply.
netstat -tulpn: print listening ports.
netstat -i: print networks interfaces.

compression

A compression is a way to compress files.

gzip <file_name>: compress a regular file; use tar with gzip for a directory tree.
tar -cvf <file_name.tar> <file_name/dir>: create tar file.
tar -cvzf <file_name.tar.gz> <file_name/dir>: create tar.gz file (gzip).
tar -xvzf <file_name.tar.gz>: extract tar file.
zip -r <file_name.zip> <file/dir>: create zip file.
unzip <file_name.zip> : extract zip file.

Process management

A process is a running instance of a program.

ps: show processes associated with the current terminal by default.
ps -ef: show a broader process listing on common Unix systems.
kill <pid>: send SIGTERM, requesting termination; it is not guaranteed to stop the process.
top: inspect processes interactively; keys vary by platform. Read its help or man top.
htop: an optional interactive terminal process viewer.

Background jobs

Bash job control manages commands started in the current interactive shell:

sleep 1000 &: start a background job.
jobs: list this shell's jobs.
Ctrl-Z: suspend the foreground job, rather than terminate it.
bg %1: continue job 1 in the background.
fg %1: continue job 1 in the foreground.
kill %1: send SIGTERM to job 1.
kill %%: signal the current job, not all background jobs.

A background command that tries to read from the terminal normally stops with SIGTTIN. Use input from a file or run it in the foreground instead.

vim

vim <file_name>: open file_name.
- i: insert mode.
- esc: normal mode.
- :w: write file.
- :q: quit file.
- :wq: write and quit file.
- :q!: quit without saving.
- :w!: write over file.
- :wq!: write over file and quit.
- :w <file_name>: write to file_name.
- /regex: search for regex.
- dd: delete line.
- yy: copy line.
- p: paste line.
- u: undo.
- ctrl + r: redo.
- gg: go to first line.
- G: go to last line.
- n: next match.
- N: previous match.
- :%s/foo/bar/g: replace every match of foo with bar throughout the buffer.
vim -R <file_name>: open file_name in read-only mode.
vim -O file1 file2: open files in vertical splits.
vim -o <file_name>: open file_name in horizontal split mode.
vim -p <file_name>: open file_name in tab mode.
vim -c <command>: execute command.
vim -c "set number" <file_name>: open file_name with line numbers.
vim -c "set number" -c "set hlsearch" <file_name>: open file_name with line numbers and highlight search.

customize the terminal

bash
zsh

References

Platzi: Curso de Introducción a la Terminal y Línea de Comandos

Python Basics

Sat, 26 Oct 2024 00:00:00 GMT

The simplicity of Python makes the language a great option for almost everyone aiming to use programming in any field. This is the reason Python is widely used in the industry, from web development, data science, machine learning, automation, scientific computing, and more. Having a huge community, and a vast number of libraries, Python is a great choice for beginners and experts alike.

In this post, we will explore some of the best features of Python, and how you can use them to your advantage.

Modules

A module is a file containing Python definitions and statements. The file name is the module name with the suffix .py appended. Within a module, the module's name (as a string) is available as the value of the global variable __name__.

A module can be imported by another program to make use of its functionality. We can define our most used functions in a module and import it, instead of copying their definitions into different programs.

if `name == "main"`

You will often see this construct in Python files. This is used to execute some code only if the file was run directly, and not imported. That is, if the file is imported, the code is not run. This is because when you import a module, the code in the module is executed, just like any other script. So if we want to have some code that we only want to run when the module is run directly, we can use this construct.

def main():
    print("Running as script!")

# execute only if run as a script
if __name__ == "__main__":
    main()

Packages

A package organizes modules and subpackages under a dotted namespace. A regular package contains __init__.py, which may be empty or contain initialization code. Namespace packages can omit that file and span multiple directories; they have been supported since Python 3.3. See the Python import reference.

# __init__.py
# In a package directory, you would use relative imports:
# from . import module1, module2, module3
print("Package initialized!")

Doing this will allow us to use namespaced modules, such as package.module1, package.module2, etc.

Scope

Scopes are the contexts in which names are looked up. There are four different scopes in Python: local, enclosing, global, and built-in.

The scope of a name defines the area of the program where you can unambiguously access that name, such as variables, functions, objects, and so on. It is determined by the place where it is declared.

Names that are declared outside of all functions are in the global scope. This means that those names can be accessed inside or outside of functions.

Names that are declared inside a function are in the local scope, and can only be accessed inside that function.

The enclosing scope is a special scope that only exists for nested functions. If the local scope is an inner or nested function, then the enclosing scope is the scope of the outer or enclosing function.

The built-in scope is the outermost scope in Python, and it is the scope that contains all of the built-in names in Python. The built-in scope is searched last, after the local, enclosing, and global scopes (LEGB (Local, Enclosing, Global, Built-in: the order in which Python looks up names)).

Figure: To resolve x inside inner(), Python checks the local scope, then the enclosing outer(), and finds x = 10 in the global scope. The search stops at the first match, so built-in is never checked. (interactive version in the post)

Example Global Scope

x = 10

def my_func():
    print(x)

my_func()
print(x)

10
10

Example Local Scope

def my_func():
    x_ = 10
    print(x_)

my_func()
try:
    print(x_)
except NameError as e:
    print(e)

10
name 'x_' is not defined

Example Enclosing Scope

def outer():
    x = 'local'

    def inner():
        nonlocal x
        x = 'nonlocal'
        print('inner:', x)

    inner()
    print('outer:', x)

outer()

inner: nonlocal
outer: nonlocal

Note that the nonlocal keyword is used to declare that x is not a local variable. Hence, when we assign a value to x inside the nested function, that change is reflected in the local variable in the enclosing function.

Example: A Local Name Shadows a Global Name

x = 10

def my_func():
    x = 20
    print(x)

my_func()
print(x)

20
10

Example Keyword `global`

x = 10

def my_func():
    global x
    x = 20
    print(x)

my_func()
print(x)

20
20

Note that the global keyword is used to declare that x is a global variable - hence, when we assign a value to x inside the function, that change is reflected when we use the value of x in the main block.

Built-in Scope

These are the names in the pre-defined built-in modules. These are always available in your Python programs. You can see the list of built-in names by typing import builtins; dir(builtins) in the Python interpreter.

Splat Operator `*` and Double Splat Operator `**`

Splat operator is a kind of unpacking operator (destructuring in JS (JavaScript)). It can be used to allow an iterable to be unpacked into positional arguments in a function call, or into a list, tuple, or set display. ** unpacks a mapping into keyword arguments or a dictionary display. These are often called the splat (*) and double splat (**) operators.

Splat Operator

def add(x, y):
    return x + y

numbers = [3, 5]
print(add(*numbers)) # 8

Double Splat Operator

def display_names(first, second):
    print(f'{first} says hello to {second}')

names = {"first": "John", "second": "Bob"}
display_names(**names) # John says hello to Bob

Multiple Arguments

def print_everything(*args):
    for count, thing in enumerate(args):
        print('{0}. {1}'.format(count, thing))

print_everything('apple', 'banana', 'cabbage')

0. apple
1. banana
2. cabbage

Keyword Arguments

def table_things(**kwargs):
    for name, value in kwargs.items():
        print('{0} = {1}'.format(name, value))

table_things(apple='fruit', cabbage='vegetable')

apple = fruit
cabbage = vegetable

Unpacking Arguments

def print_three_things(a, b, c):
    print('a = {0}, b = {1}, c = {2}'.format(a, b, c))

my_list = ['aardvark', 'baboon', 'cat']
print_three_things(*my_list)

my_dict = {'a': 'apple', 'b': 'banana', 'c': 'cherry'}
print_three_things(**my_dict)

a = aardvark, b = baboon, c = cat
a = apple, b = banana, c = cherry

Positional and keyword arguments (*args, **kwargs)

def func(*args, **kwargs):
    print(args)
    print(kwargs)

func(1, 2, 3, 4, 5, 6, 7, 8, 9, 10, a=1, b=2, c=3)

(1, 2, 3, 4, 5, 6, 7, 8, 9, 10)
{'a': 1, 'b': 2, 'c': 3}

Yeison Liscano

Pruebas de hipótesis paso a paso: una diferencia no basta

La pregunta debe convertirse en una afirmación medible

El diseño y la muestra se deciden antes del resultado

La prueba depende de los datos y de cómo se obtuvieron

El valor p mide la incompatibilidad con la hipótesis nula

La significancia debe acompañarse del efecto y su incertidumbre

Buscar significancia hasta encontrarla altera la prueba

La conclusión debe responder la pregunta y declarar sus límites

Referencias

5 Pourquoi : le cas d'école du Jefferson Memorial n'a pas la fin qu'on raconte, et comment chercher plus d'une cause

Le cas d'école tient en une chaîne, une action et un chiffre

Les archives montrent d'autres causes, un autre mémorial, un essai abandonné

Une chaîne de pourquoi choisit une branche et cache les autres

GitLab et AWS ont corrigé les conditions, pas la personne

S'arrêter sur une personne ou sur un manque ferme l'analyse

L'arbre des causes remplace « pourquoi ? » par « qu'a-t-il fallu ? »

Un script repère les pourquoi qui s'arrêtent trop tôt

Une analyse se juge à la force de ses actions, pas au nombre de pourquoi

Au mémorial, il manquait une question, pas un pourquoi de plus

Références

PDCA et IA : vérifier les gains avant de généraliser

Le PDCA transforme une amélioration supposée en hypothèse à tester

Planifier commence par un diagnostic et une mesure de référence

Faire exige de repenser le processus et les responsabilités

Vérifier mesure le service rendu et les effets sur chaque groupe

Agir conduit à généraliser, corriger ou arrêter

Un premier cycle commence par une fiche de décision

Références

5W2H (QQOQCCP) : sept questions qui rendent un plan d'action exécutable, et celle qui a trois sens

Une action sans responsable ni date n'est qu'un souhait

Les questions ont 2 300 ans ; le « combien » n'a pas d'origine connue

Le même tableau sert à décrire un problème ou à planifier une action

Le « pourquoi » a trois sens selon le moment où on le pose

Dans le PDCA, le 5W2H vient après l'analyse des causes, pas avant

Réécrire le plan ligne par ligne fait apparaître ce qui manque

La grille ne priorise pas, et rien ne prouve son effet

Une action exécutable nomme une personne, une date et un coût

Références

ASGI Event by Event: How scope, receive and send Keep a Connection Open, and Why an Async WSGI Would Not

WSGI hands the application one request and takes back one response

An ASGI application is called once per connection, with three arguments

Every event is a dict, and its type key names its shape

A body can arrive in several events, so read until more_body is false

receive() is also how the application learns the client has left

A WebSocket needs a second coroutine to listen and push at once

Lifespan events give the application a start and an end of its own

WSGI code runs on a thread beside the event loop, one request at a time

Async was never the missing piece; a way in for later events was

References

ROI logiciel : mesurer les gains sans surestimer les économies

Le même logiciel a un ROI différent pour le client et le fournisseur

La formule exige une période et un scénario de référence

Le coût dépasse l’achat ou le développement initial

Les gains reposent sur la marge et les économies réellement obtenues

Une amélioration doit pouvoir être attribuée au logiciel

Le cas chiffré révèle les hypothèses qui font basculer le ROI

La décision dépend aussi du délai, du risque et des résultats observés

Références

Perfect Secrecy and Kerckhoffs' Principle: How a Provably Unbreakable Cipher Was Broken

Venona read Soviet traffic that was protected by one-time pads

Perfect secrecy means the ciphertext does not change the attacker's odds

The one-time pad is perfectly secret, and the proof takes one line

Perfect secrecy requires at least as many keys as messages

A reused key breaks the one-message hypothesis, not the proof

Kerckhoffs' principle is built into the definition

Kerckhoffs also accepted that perfect secrecy is too expensive

A provably unbreakable cipher breaks when one of its hypotheses fails

References

Análisis de datos: los cuatro tipos y qué información usa cada uno

1. Análisis descriptivo: ¿qué está sucediendo?

2. Análisis predictivo: ¿qué va a suceder?

3. Análisis diagnóstico: ¿por qué sucedió esto?

4. Análisis prescriptivo: ¿qué debo hacer?

El orden correcto depende de qué se quiere medir

Cada fase entrega lo que la siguiente necesita

Método científico aplicado al desarrollo de software y la seguridad

El ciclo, traducido al trabajo diario

Dónde aparece en la práctica

Threat models como conjuntos de hipótesis

Every event is a dict, and its `type` key names its shape

A body can arrive in several events, so read until `more_body` is false

`receive()` is also how the application learns the client has left