I'm betting that pilgrim he met was Carlton, who likely had his sword arm removed or something of that sort.