English is an umbrella term for a large number of regional variations on a language that results from the cultural collision of a group of closely related Germano-Scandinavian languages with a Romantic language (Norman French), beginning in the 11th Century in what is now England. It continues to develop, but took several centuries to reach it's modern form. Later, through trade, imperialism, and colonialism it became widely established around the world. It was always a language with many variants and dialects and continues to be so, even or perhaps especially, in England. For my purposes I'm going to look at the language at a level that largely obscures the variations. But those variations are real and important at other levels and I will refer to them at times.
English has considerable leeway in word order, but if we examine a large number of sentences, we can discover conventions which apply most of the time. Mostly English speakers use the order: subject, verb, object, but there is considerable variation. Consider: "Is it you?" (VSO); or "words fail me" (OVS). In Star Wars Yoda's English consistently breaks the rules, but is perfectly comprehensible as English.
Influenced by a cult of grammar spawned by the European obsession with Classical Greece, the Roman Empire, and the 18th Century discovery of Pāṇini's treatise on grammar, the Aṣṭādhyāyī, generations of scholars dissected the English language to discover patterns of use that we now think of as the "rules" of English grammar. And they attempted to standardise written English, though they seem to have done a very bad job of this.
There are two tendencies in the study of grammar. One tendency is to think that if one can discover rules then they ought to be followed, we call this the prescriptivist tendency. Unfortunately for English speakers, the cult of grammar did not just discover rules, they also invented arbitrary rules, like not splitting infinitives or leaving prepositions dangling. In spoken English, people split infinitives all the time. There is no natural rule against doing so, i.e. no rule that emerges from how people speak English. Even so the prescriptivist tendency insists that such rules must be followed, and proponents produce turgid and unnatural sounding speech as a result.
By contrast, descriptivist tendency acknowledges that there are conventions about how to speak English, but concedes that the variations are valid forms of English and that the leeway allows English speakers to make novel constructions if they wish. Yoda's English is still English. Descriptivists see their role as grammarians as describing how people use language, rather than being judges of good and bad English by some arbitrary standard. And they tend to reject the underlying power structures in such standards, which are often related to class and privilege.
So there are rules that govern spoken English and most of the time when we speak we follow these rules. But its apparent that we can break the rules and still be understood to be speaking English. It's also apparent that almost no one (except obnoxious pedants) is consciously parsing their sentences and making conscious decisions about vocabulary or word order; when we speak we may obey rules, but we are not using rules to construct speech. Indeed a good deal of humour arises from breaking the rules of speech in subtle ways - puns, substitutions, and other plays on words. Similarly in poetry rhyme and rhythm may override syntax and grammar.
How does this work? Are we unconsciously following the rules and then sometimes breaking them? Probably not. Instead what happens is that as we grow up we learn first vocabulary and then syntax and grammar. It's like any skill. As we learn a sport, or to drive a car, or to do handwriting, we start off consciously applying rules. At first we are slow and clumsy. But gradually we develop competencies that mean the rules fade into the background and we become fluent. John Searle describes this as developing dispositions to behaviour that is consistent with the rules, but which does not itself follow the rules either consciously or unconsciously.
This makes good sense to me. Following rules is too slow, even if the rule following is happening unconsciously. Parsing a sentence in Sanskrit can take me a long time because I have never developed the fluency that comes with leaving the rules behind and internalising the script, morphology, syntax and grammar of the language. I can understand written Sanskrit with some effort, but not speak the language. Just as I can do other skills that I am competent at or have mastered. When associated physical skills, such as playing the guitar, we call this muscle memory - my hands just know where to go to play certain chords or patterns; some songs remain accessible to my hands even when I struggle to consciously recall how they are played.
This result is not good for the people who think of the brain as a computer. Computers can only follow rules. When they get powerful enough they can give rule following a certain grace, but they are not doing what we are doing. They do not, and at present cannot, develop a disposition to behaviour that is consistent with the rules. Computers are bound by rules in ways that human beings are not. We can deliberately cheat, for example. Or we can try to distract our opponent. Or we can appreciate that our opponent has made a particularly good or bad move. People say that the computer can "play" chess, but all it does is calculate chess moves very fast. The verb play does not apply here. The computer does not even move its own pieces. Arguably powerful neural networks that are tuned to do one activity, like calculate chess moves, might be approaching this capacity to develop rule following dispositions. However, to date no computer has needed to be programmed with correct use of the word j'adoube - which a player says when they idly touch a piece they do not intend to move.
Consider a sportsman playing a the highest level in a team sport. There are a number of rules which govern the conduct of the players and the progress of the game at any given moment. The players have usually played the game from an early age. They know very well what the rules are. If they were simply following the rules there would be no need for an umpire or referee. Of course players may deliberately break the rules because it is to their advantage if they do not get caught. If they are caught they are penalised, so there maybe an element of calculation in this. In many sports the best players have a disposition to exploit greys areas (in soccer at the point of tackling; in rugby the off-side rules; the charging rules in basketball; and so on). But the majority of the penalties are not given for deliberate fouls, but for mistakes.
We make mistakes because we are not following rules, instead relying on our behaviour to be rule consistent while pursuing the goal of the activity without any direct reference to the relevant rules. The same happens with speech. Slips of the tongue may go unnoticed, be hilarious, or confusing. It also happens when we visit a foreign country and find the rules for social intercourse are different from what we grew up with.
A sportsman making a heroic effort to kick a ball between two sticks, is completely focussed on that goal: they are not thinking about how to run, how to control the ball, how to kick, how to aim, how to avoid the opposing team. If all this was happening in real time, at a sprint down the field, under pressure from the other team, and if we were relying on rules, it would all come crashing down. And we do see this in amateur games. Where there is less skill and someone has to think about what they are doing, even unconsciously, they are less efficient, less effective.
We all know that the master of anything makes it look effortless. In a sense it is effortless. When Michael Jordon would sail into the air, float there for a second or two, and slam the ball down into the hoop, only to land gracefully on his feet, all seemingly in slow motion, it was astounding. He did it again and again. Think about everything he was coordinating in those moments: his whole body had to be coordinated in just the right way, he had to exquisitely accurately judge where everything was in space: himself, the hoop, the other players; and all at the top speed at which a very fit, 1.98 metre tall man could sprint.
Does anyone genuinely think that because they can discover patterns in what Michael Jordon was doing in this video, that this means he was following rules? Please.
So yes, there are rules; or at least rules can be discovered. And yes our behaviour is tuned to harmonise with those rules. But no, we do not consciously or unconsciously follow these rules. Instead after a period of learning the rules, we adapt our behavioural range to be more or less compliant with the rules, and to exploit grey areas to our advantage. Anti-social behaviour is another question all together that I'll have to deal with separately.
But here's the thing. This observation applies to all kinds of agents doing or causing all kinds of actions. The atoms does not think, "Oh I'm experiencing the curvature of space, now I have to lean to the left"! The atom has a disposition to follow the curvature of space. We can explain the behaviour, but not the disposition in this case. The universe is just like that. A eukaryote cell undergoing mitosis does not follow rules either - it has no mechanism for following rules. But the process does follow a pattern. Rules can be discovered in nature at all levels. Science is all about finding and describing these rules. Atoms must follow the rules that govern them. But as we go up the hierarchy of scale and complexity, flexibility emerges as a property. A cell also follows patterns, but it has vastly more degrees of freedom than an atom does. Its behaviour is more sophisticated given the circumstances. But also because it is interacting with other complex entities on its own level (i.e. other cells) the cell experiences a vastly greater diversity of circumstances than an atom does.
Patterns can be discovered in human behaviour at various levels. We might begin by learning rules, but we rapidly progress to developing dispositions to behave in ways that are rule-consistent without actually being rule-determined. This may be why certain behaviours developed early on and constantly reinforced might be hard to shift. It explains not only our incredible successes, but also our inexplicable failures. If we simply followed rules, we'd be computer-like in our accuracy. But we are not. We effortlessly produce rule-consistent behaviour, though our levels of consistency may vary.