Commit Graph

112 Commits

Author SHA1 Message Date
Vineet Gupta
6c00350b57 ARC: Workaround spinlock livelock in SMP SystemC simulation
Some ARC SMP systems lack native atomic R-M-W (LLOCK/SCOND) insns and
can only use atomic EX insn (reg with mem) to build higher level R-M-W
primitives. This includes a SystemC based SMP simulation model.

So rwlocks need to use a protecting spinlock for atomic cmp-n-exchange
operation to update reader(s)/writer count.

The spinlock operation itself looks as follows:

	mov reg, 1		; 1=locked, 0=unlocked
retry:
	EX reg, [lock]		; load existing, store 1, atomically
	BREQ reg, 1, rety	; if already locked, retry

In single-threaded simulation, SystemC alternates between the 2 cores
with "N" insn each based scheduling. Additionally for insn with global
side effect, such as EX writing to shared mem, a core switch is
enforced too.

Given that, 2 cores doing a repeated EX on same location, Linux often
got into a livelock e.g. when both cores were fiddling with tasklist
lock (gdbserver / hackbench) for read/write respectively as the
sequence diagram below shows:

           core1                                   core2
         --------                                --------
1. spin lock [EX r=0, w=1] - LOCKED
2. rwlock(Read)            - LOCKED
3. spin unlock  [ST 0]     - UNLOCKED
                                         spin lock [EX r=0,w=1] - LOCKED
                      -- resched core 1----

5. spin lock [EX r=1] - ALREADY-LOCKED

                      -- resched core 2----
6.                                       rwlock(Write) - READER-LOCKED
7.                                       spin unlock [ST 0]
8.                                       rwlock failed, retry again

9.                                       spin lock  [EX r=0, w=1]
                      -- resched core 1----

10  spinlock locked in #9, retry #5
11. spin lock [EX gets 1]
                      -- resched core 2----
...
...

The fix was to unlock using the EX insn too (step 7), to trigger another
SystemC scheduling pass which would let core1 proceed, eliding the
livelock.

Signed-off-by: Vineet Gupta <vgupta@synopsys.com>
2013-09-27 16:28:48 +05:30
Vineet Gupta
0752adfda1 ARC: Fix 32-bit wrap around in access_ok()
Anton reported

 | LTP tests syscalls/process_vm_readv01 and process_vm_writev01 fail
 | similarly in one testcase test_iov_invalid -> lvec->iov_base.
 | Testcase expects errno EFAULT and return code -1,
 | but it gets return code 1 and ERRNO is 0 what means success.

Essentially test case was passing a pointer of -1 which access_ok()
was not catching. It was doing [@addr + @sz <= TASK_SIZE] which would
pass for @addr == -1

Fixed that by rewriting as [@addr <= TASK_SIZE - @sz]

Reported-by: Anton Kolesov <Anton.Kolesov@synopsys.com>
Signed-off-by: Vineet Gupta <vgupta@synopsys.com>
2013-09-27 16:28:47 +05:30
Noam Camus
c3567f8a35 ARC: SMP failed to boot due to missing IVT setup
Commit 05b016ecf5 "ARC: Setup Vector Table Base in early boot" moved
the Interrupt vector Table setup out of arc_init_IRQ() which is called
for all CPUs, to entry point of boot cpu only, breaking booting of others.

Fix by adding the same to entry point of non-boot CPUs too.

read_arc_build_cfg_regs() printing IVT Base Register didn't help the
casue since it prints a synthetic value if zero which is totally bogus,
so fix that to print the exact Register.

[vgupta: Remove the now stale comment from header of arc_init_IRQ and
also added the commentary for halt-on-reset]

Cc: Gilad Ben-Yossef <gilad@benyossef.com>
Cc: Cc: <stable@vger.kernel.org> #3.11
Signed-off-by: Noam Camus <noamc@ezchip.com>
Signed-off-by: Vineet Gupta <vgupta@synopsys.com>
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
2013-09-12 07:40:08 -07:00
Vineet Gupta
07b9b65147 ARC: fix new Section mismatches in build (post __cpuinit cleanup)
--------------->8--------------------
WARNING: vmlinux.o(.text+0x708): Section mismatch in reference from the
function read_arc_build_cfg_regs() to the function
.init.text:read_decode_cache_bcr()

WARNING: vmlinux.o(.text+0x702): Section mismatch in reference from the
function read_arc_build_cfg_regs() to the function
.init.text:read_decode_mmu_bcr()
--------------->8--------------------

Signed-off-by: Vineet Gupta <vgupta@synopsys.com>
2013-09-05 19:19:06 +05:30
Mischa Jonker
7efd0da2d1 ARC: Fix __udelay calculation
Cast usecs to u64, to ensure that the (usecs * 4295 * HZ)
multiplication is 64 bit.

Initially, the (usecs * 4295 * HZ) part was done as a 32 bit
multiplication, with the result casted to 64 bit. This led to some bits
falling off, causing a "DMA initialization error" in the stmmac Ethernet
driver, due to a premature timeout.

Signed-off-by: Mischa Jonker <mjonker@synopsys.com>
Signed-off-by: Vineet Gupta <vgupta@synopsys.com>
2013-09-05 10:31:12 +05:30
Mischa Jonker
6532b02fe5 ARC: Add read*_relaxed to asm/io.h
Some drivers require these, and ARC didn't had them yet.

Signed-off-by: Mischa Jonker <mjonker@synopsys.com>
Signed-off-by: Vineet Gupta <vgupta@synopsys.com>
2013-09-05 10:31:11 +05:30
Vineet Gupta
947bf103fc ARC: [ASID] Track ASID allocation cycles/generations
This helps remove asid-to-mm reverse map

While mm->context.id contains the ASID assigned to a process, our ASID
allocator also used asid_mm_map[] reverse map. In a new allocation
cycle (mm->ASID >= @asid_cache), the Round Robin ASID allocator used this
to check if new @asid_cache belonged to some mm2 (from prev cycle).
If so, it could locate that mm using the ASID reverse map, and mark that
mm as unallocated ASID, to force it to refresh at the time of switch_mm()

However, for SMP, the reverse map has to be maintained per CPU, so
becomes 2 dimensional, hence got rid of it.

With reverse map gone, it is NOT possible to reach out to current
assignee. So we track the ASID allocation generation/cycle and
on every switch_mm(), check if the current generation of CPU ASID is
same as mm's ASID; If not it is refreshed.

(Based loosely on arch/sh implementation)

Signed-off-by: Vineet Gupta <vgupta@synopsys.com>
2013-08-30 21:42:19 +05:30
Vineet Gupta
c60115537c ARC: [ASID] activate_mm() == switch_mm()
ASID allocation changes/2

Use the fact that switch_mm() and activate_mm() are exactly same code
now while acknowledging the semantical difference in comment

Signed-off-by: Vineet Gupta <vgupta@synopsys.com>
2013-08-30 21:42:19 +05:30
Vineet Gupta
3daa48d1d9 ARC: [ASID] get_new_mmu_context() to conditionally allocate new ASID
ASID allocation changes/1

This patch does 2 things:

(1) get_new_mmu_context() NOW moves mm->ASID to a new value ONLY if it
    was from a prev allocation cycle/generation OR if mm had no ASID
    allocated (vs. before would unconditionally moving to a new ASID)

    Callers desiring unconditional update of ASID, e.g.local_flush_tlb_mm()
    (for parent's address space invalidation at fork) need to first force
    the parent to an unallocated ASID.

(2) get_new_mmu_context() always sets the MMU PID reg with unchanged/new
    ASID value.

The gains are:
- consolidation of all asid alloc logic into get_new_mmu_context()
- avoiding code duplication in switch_mm() for PID reg setting
- Enables future change to fold activate_mm() into switch_mm()

Signed-off-by: Vineet Gupta <vgupta@synopsys.com>
2013-08-30 21:42:18 +05:30
Vineet Gupta
5bd87adf9b ARC: [ASID] Refactor the TLB paranoid debug code
-Asm code already has values of SW and HW ASID values, so they can be
 passed to the printing routine.

Signed-off-by: Vineet Gupta <vgupta@synopsys.com>
2013-08-30 21:42:18 +05:30
Vineet Gupta
ade922f8e2 ARC: [ASID] Remove legacy/unused debug code
Signed-off-by: Vineet Gupta <vgupta@synopsys.com>
2013-08-30 21:42:17 +05:30
Vineet Gupta
483e9bcb01 ARC: MMUv4 preps/3 - Abstract out TLB Insert/Delete
This reorganizes the current TLB operations into psuedo-ops to better
pair with MMUv4's native Insert/Delete operations

Signed-off-by: Vineet Gupta <vgupta@synopsys.com>
2013-08-30 10:22:48 +05:30
Vineet Gupta
d091fcb97f ARC: MMUv4 preps/2 - Reshuffle PTE bits
With previous commit freeing up PTE bits, reassign them so as to:

- Match the bit to H/w counterpart where possible
  (e.g. MMUv2 GLOBAL/PRESENT, this avoids a shift in create_tlb())
- Avoid holes in _PAGE_xxx definitions

Signed-off-by: Vineet Gupta <vgupta@synopsys.com>
2013-08-30 10:19:12 +05:30
Vineet Gupta
64b703ef27 ARC: MMUv4 preps/1 - Fold PTE K/U access flags
The current ARC VM code has 13 flags in Page Table entry: some software
(accesed/dirty/non-linear-maps) and rest hardware specific. With 8k MMU
page, we need 19 bits for addressing page frame so remaining 13 bits is
just about enough to accomodate the current flags.

In MMUv4 there are 2 additional flags, SZ (normal or super page) and WT
(cache access mode write-thru) - and additionally PFN is 20 bits (vs. 19
before for 8k). Thus these can't be held in current PTE w/o making each
entry 64bit wide.

It seems there is some scope of compressing the current PTE flags (and
freeing up a few bits). Currently PTE contains fully orthogonal distinct
access permissions for kernel and user mode (Kr, Kw, Kx; Ur, Uw, Ux)
which can be folded into one set (R, W, X). The translation of 3 PTE
bits into 6 TLB bits (when programming the MMU) can be done based on
following pre-requites/assumptions:

1. For kernel-mode-only translations (vmalloc: 0x7000_0000 to
   0x7FFF_FFFF), PTE additionally has PAGE_GLOBAL flag set (and user
   space entries can never be global). Thus such a PTE can translate
   to Kr, Kw, Kx (as appropriate) and zero for User mode counterparts.

2. For non global entries, the PTE flags can be used to create mirrored
   K and U TLB bits. This is true after commit a950549c67
   "ARC: copy_(to|from)_user() to honor usermode-access permissions"
   which ensured that user-space translations _MUST_ have same access
   permissions for both U/K mode accesses so that  copy_{to,from}_user()
   play fair with fault based CoW break and such...

There is no such thing as free lunch - the cost is slightly infalted
TLB-Miss Handlers.

Signed-off-by: Vineet Gupta <vgupta@synopsys.com>
2013-08-29 17:51:36 +05:30
Vineet Gupta
4b06ff35fb ARC: Code cosmetics (Nothing semantical)
* reduce editor lines taken by pt_regs
* ARCompact ISA specific part of TLB Miss handlers clubbed together
* cleanup some comments

Signed-off-by: Vineet Gupta <vgupta@synopsys.com>
2013-08-29 17:51:15 +05:30
Vineet Gupta
fce16bc35a ARC: Entry Handler tweaks: Optimize away redundant IRQ_DISABLE_SAVE
In the exception return path, for both U/K cases, intr are already
disabled (for various existing reasons). So when we drop down to
@restore_regs, we need not redo that.

There was subtle issue - when intr were NOT being disabled for
ret-to-kernel-but-no-preemption case - now fixed by moving the
IRQ_DISABLE further up in @resume_kernel_mode.

So what do we gain:

* Shaves off a few insn in return path.

* Eliminates the need for IRQ_DISABLE_SAVE assembler macro for ARCv2
  hence allows for entry code sharing.

Signed-off-by: Vineet Gupta <vgupta@synopsys.com>
2013-08-26 09:40:25 +05:30
Vineet Gupta
37f3ac498c ARC: Exception Handlers Code consolidation
After the recent cleanups, all the exception handlers now have same
boilerplate prologue code. Move that into common macro.

This reduces readability but helps greatly with sharing / duplicating
entry code with ARCv2 ISA where the handlers are pretty much the same,
just the entry prologue is different (due to hardware assist).

Also while at it, add the missing FAKE_RET_FROM_EXCPN calls in couple of
places to drop down to pure kernel mode (from exception mode) before
jumping off into "C" code.

Signed-off-by: Vineet Gupta <vgupta@synopsys.com>
2013-08-26 09:40:25 +05:30
Vineet Gupta
4ffd9e2c4d ARC: SMP build breakage
Signed-off-by: Vineet Gupta <vgupta@synopsys.com>
2013-07-26 15:34:22 -07:00
Linus Torvalds
76d3f4c27d ARC changes for 3.11
Highlights of changes:
 
 -Continuation of ARC MM changes from 3.10 including
    zero page optimization;
    Setting pagecache pages dirty by default;
    Non executable stack by default;
    Reducing dcache flushes for aliasing VIPT config
 
 -Long overdue rework of pt_regs machinery - removing the unused word gutters
  and adding ECR register to baseline (helps cleanup lot of low level code)
 
 -Support for ARC gcc 4.8
 
 -Few other preventive fixes, cosmetics, usage of Kconfig helper..
 
 The diffstat is larger than normal primarily because of arcregs.h header split
 as well as beautification of macros in entry.h
 -----BEGIN PGP SIGNATURE-----
 Version: GnuPG v1.4.12 (GNU/Linux)
 
 iQIcBAABAgAGBQJR0mEbAAoJEGnX8d3iisJe0lEP+gPtakLaOBprpxrVttK3DCLw
 E28LMVdR+33jfqugqk8xMVQh9gc5mvSGGJkZFGPoQX9gumdDjJP4vRxqlZxLgpNH
 siNr+LvyzUOUa3bQrB6q639jlqlvPcKO19Yjzx1vqKRkMBsV8+YG4+8tdBNvh2K4
 dr6nB6gqmZiCWmuUpTQ+D2J7vjlNwi5PWHBmlP0uFH433Z78Wk2tT2LWUHihl6Do
 +Uhp/Ldu7EDFTVLDlqc5pv0EUNnGzuR9dpN3XuxfZLknKMtJKpwqLq0/6bnfWqck
 g+Cmh8jvj5SAutbzZiG/xggZMtSHOXsqY06+MALquQ7tcoXrm7U0Ubbplt1B55y/
 OW8yWD/b1S+2pVVCmbIWV2EinNmSMpdX18nVUmWSggQYBjKQnMq611Y7q1EGBmen
 cTCY9u7jv7tD+Qem4H/rze7b+EKlcSFDhzL/Df/iHHnvAmEjb+trmD3qJ6PtA93w
 24b+fVaBi5j5dHZx4NEggDKTkmUVcqkD2qfg4Ckg4mqq7cBHfJponuoDJX4KM7t1
 qozSpKLMMSzRd3L7DQhePqNJfrlOD2rhD4ELRCJA131dUANr4Wpi2yvIASro135A
 4gmpYX5WtjohWjL73aRCwsRcLTnyNmqjPfibT35ZaAygW7wXII7zNVzKY09oEPN3
 keol0YTXDokCzGN432NI
 =65Fq
 -----END PGP SIGNATURE-----

Merge tag 'arc-v3.11-rc1-part1' of git://git.kernel.org/pub/scm/linux/kernel/git/vgupta/arc

Pull first batch of ARC changes from Vineet Gupta:
 "There's a second bunch to follow next week - which depends on commits
  on other trees (irq/net).  I'd have preferred the accompanying ARC
  change via respective trees, but it didn't workout somehow.

  Highlights of changes:

   - Continuation of ARC MM changes from 3.10 including

       zero page optimization
       Setting pagecache pages dirty by default
       Non executable stack by default
       Reducing dcache flushes for aliasing VIPT config

   - Long overdue rework of pt_regs machinery - removing the unused word
     gutters and adding ECR register to baseline (helps cleanup lot of
     low level code)

   - Support for ARC gcc 4.8

   - Few other preventive fixes, cosmetics, usage of Kconfig helper..

  The diffstat is larger than normal primarily because of arcregs.h
  header split as well as beautification of macros in entry.h"

* tag 'arc-v3.11-rc1-part1' of git://git.kernel.org/pub/scm/linux/kernel/git/vgupta/arc: (32 commits)
  ARC: warn on improper stack unwind FDE entries
  arc: delete __cpuinit usage from all arc files
  ARC: [tlb-miss] Fix bug with CONFIG_ARC_DBG_TLB_MISS_COUNT
  ARC: [tlb-miss] Extraneous PTE bit testing/setting
  ARC: Adjustments for gcc 4.8
  ARC: Setup Vector Table Base in early boot
  ARC: Remove explicit passing around of ECR
  ARC: pt_regs update #5: Use real ECR for pt_regs->event vs. synth values
  ARC: stop using pt_regs->orig_r8
  ARC: pt_regs update #4: r25 saved/restored unconditionally
  ARC: K/U SP saved from one location in stack switching macro
  ARC: Entry Handler tweaks: Simplify branch for in-kernel preemption
  ARC: Entry Handler tweaks: Avoid hardcoded LIMMS for ECR values
  ARC: Increase readability of entry handlers
  ARC: pt_regs update #3: Remove unused gutter at start of callee_regs
  ARC: pt_regs update #2: Remove unused gutter at start of pt_regs
  ARC: pt_regs update #1: Align pt_regs end with end of kernel stack page
  ARC: pt_regs update #0: remove kernel stack canary
  ARC: [mm] Remove @write argument to do_page_fault()
  ARC: [mm] Make stack/heap Non-executable by default
  ...
2013-07-03 11:09:27 -07:00
Al Viro
40d158e618 consolidate io_remap_pfn_range definitions
Signed-off-by: Al Viro <viro@zeniv.linux.org.uk>
2013-06-29 12:46:35 +04:00
Paul Gortmaker
ce7599567e arc: delete __cpuinit usage from all arc files
The __cpuinit type of throwaway sections might have made sense
some time ago when RAM was more constrained, but now the savings
do not offset the cost and complications.  For example, the fix in
commit 5e427ec2d0 ("x86: Fix bit corruption at CPU resume time")
is a good example of the nasty type of bugs that can be created
with improper use of the various __init prefixes.

After a discussion on LKML[1] it was decided that cpuinit should go
the way of devinit and be phased out.  Once all the users are gone,
we can then finally remove the macros themselves from linux/init.h.

Note that some harmless section mismatch warnings may result, since
notify_cpu_starting() and cpu_up() are arch independent (kernel/cpu.c)
are flagged as __cpuinit  -- so if we remove the __cpuinit from
arch specific callers, we will also get section mismatch warnings.
As an intermediate step, we intend to turn the linux/init.h cpuinit
content into no-ops as early as possible, since that will get rid
of these warnings.  In any case, they are temporary and harmless.

This removes all the arch/arc uses of the __cpuinit macros from
all C files.  Currently arc does not have any __CPUINIT used in
assembly files.

[1] https://lkml.org/lkml/2013/5/20/589

Cc: Vineet Gupta <vgupta@synopsys.com>
Signed-off-by: Paul Gortmaker <paul.gortmaker@windriver.com>
Signed-off-by: Vineet Gupta <vgupta@synopsys.com>
2013-06-27 14:37:58 +05:30
Vineet Gupta
38a9ff6d24 ARC: Remove explicit passing around of ECR
With ECR now part of pt_regs

* No need to propagate from lowest asm handlers as arg
* No need to save it in tsk->thread.cause_code
* Avoid bit chopping to access the bit-fields

More code consolidation, cleanup

Signed-off-by: Vineet Gupta <vgupta@synopsys.com>
2013-06-26 15:30:50 +05:30
Vineet Gupta
502a0c775c ARC: pt_regs update #5: Use real ECR for pt_regs->event vs. synth values
pt_regs->event was set with artificial values to identify the low level
system event (syscall trap / breakpoint trap / exceptions / interrupts)

With r8 saving out of the way, the full word can be used to save real
ECR (Exception Cause Register) which helps idenify the event naturally,
including additional info such as cause code, param.
Only for Interrupts, where ECR is not applicable, do we resort to
synthetic non ECR values.

SAVE_ALL_TRAP/EXCEPTIONS can now be merged as they both use ECR with
different runtime values.

The ptrace helpers now use the sub-fields of ECR to distinguish the
events (e.g. vector 0x25 is trap, param 0 is syscall...)

The following benefits will follow:

(1) This centralizes the location of where ECR is saved and will allow
    the cleanup of task->thread.cause_code ECR placeholder which is set
    in non-uniform way. Then ARC VM code can safely rely on it being
    there for purpose of finer grained VM_EXEC dcache flush (based on
    exec fault: I-TLB Miss)

(2) Further, ECR being passed around from low level handlers as arg can
    be eliminated as it is part of standard reg-file in pt_regs

Signed-off-by: Vineet Gupta <vgupta@synopsys.com>
2013-06-26 14:04:48 +05:30
Vineet Gupta
352c1d95e3 ARC: stop using pt_regs->orig_r8
Historically, pt_regs have had orig_r8, an overloaded container for
  (1) backup copy of r8 (syscall number Trap Exceptions)
  (2) additional system state: (syscall/Exception/Interrupt)

There is no point in keeping (1) since syscall number is never clobbered
in-place, in pt_regs, unlike r0 which duals as first syscall arg as well
as syscall return value and in case of syscall restart, the orig arg0
needs restoring (from orig_r0)  after having been updated in-place with
syscall ret value.

This further paves way to convert (2) to contain ECR itself (rather than
current madeup values)

Signed-off-by: Vineet Gupta <vgupta@synopsys.com>
2013-06-22 19:23:26 +05:30
Vineet Gupta
359105bdb0 ARC: pt_regs update #4: r25 saved/restored unconditionally
(This is a VERY IMP change for low level interrupt/exception handling)

-----------------------------------------------------------------------
WHAT
-----------------------------------------------------------------------
* User 25 now saved in pt_regs->user_r25 (vs. tsk->thread_info.user_r25)

* This allows Low level interrupt code to unconditionally save r25
  (vs. the prev version which would only do it for U->K transition).
  Ofcourse for nested interrupts, only the pt_regs->user_r25 of
  bottom-most frame is useful.

* simplifies the interrupt prologue/epilogue

* Needed for ARCv2 ISA code and done here to keep design similar with
  ARCompact event handling

-----------------------------------------------------------------------
WHY
-------------------------------------------------------------------------
With CONFIG_ARC_CURR_IN_REG, r25 is used to cache "current" task pointer
in kernel mode. So when entering kernel mode from User Mode
- user r25 is specially safe-kept (it being a callee reg is NOT part of
  pt_regs which are saved by default on each interrupt/trap/exception)
- r25 loaded with current task pointer.

Further, if interrupt was taken in kernel mode, this is skipped since we
know that r25 already has valid "current" pointer.

With 2 level of interrupts in ARCompact ISA, detecting this is difficult
but still possible, since we could be in kernel mode but r25 not already saved
(in fact the stack itself might not have been switched).

A. User mode
B. L1 IRQ taken
C. L2 IRQ taken (while on 1st line of L1 ISR)

So in #C, although in kernel mode, r25 not saved (infact SP not
switched at all)

Given that ARcompact has manual stack switching, we could use a bit of
trickey - The low level code would make sure that SP is only set to kernel
mode value at the very end (after saving r25). So a non kernel mode SP,
even if in kernel mode, meant r25 was NOT saved.

The same paradigm won't work in ARCv2 ISA since SP is auto-switched so
it's setting can't be delayed/constrained.

Signed-off-by: Vineet Gupta <vgupta@synopsys.com>
2013-06-22 19:23:25 +05:30
Vineet Gupta
ba3558c772 ARC: K/U SP saved from one location in stack switching macro
This paves way for further simplifications.

There's an overhead of 1 insn for the non-common case of interrupt taken
from kernel mode.

Signed-off-by: Vineet Gupta <vgupta@synopsys.com>
2013-06-22 19:23:25 +05:30
Vineet Gupta
1898a959b7 ARC: Entry Handler tweaks: Avoid hardcoded LIMMS for ECR values
Signed-off-by: Vineet Gupta <vgupta@synopsys.com>
2013-06-22 19:23:23 +05:30
Vineet Gupta
3ebedbb2fd ARC: Increase readability of entry handlers
* use artificial PUSH/POP contructs for CORE Reg save/restore to stack
* use artificial PUSHAX/POPAX contructs for Auxiliary Space regs
* macro'ize multiple copies of callee-reg-save/restore (SAVE_R13_TO_R24)
* use BIC insn for inverse-and operation

Signed-off-by: Vineet Gupta <vgupta@synopsys.com>
2013-06-22 19:23:23 +05:30
Vineet Gupta
16f9afe651 ARC: pt_regs update #3: Remove unused gutter at start of callee_regs
This is trickier than prev two:

* context switching code saves kernel mode callee regs in the format of
  struct callee_regs thus needs adjustment. This also reduces the height
  of topmost kernel stack frame by 1 word.

* Since kernel stack unwinder is sensitive to height of topmost kernel
  stack frame, that needs a word of adjustment too.

ptrace needs a bit of updating since pt_regs now diverges from
user_regs_struct.

Signed-off-by: Vineet Gupta <vgupta@synopsys.com>
2013-06-22 19:23:22 +05:30
Vineet Gupta
2fa919045b ARC: pt_regs update #2: Remove unused gutter at start of pt_regs
Signed-off-by: Vineet Gupta <vgupta@synopsys.com>
2013-06-22 19:23:22 +05:30
Vineet Gupta
283237a04f ARC: pt_regs update #1: Align pt_regs end with end of kernel stack page
Historically, pt_regs would end at offset of 1 word from end of stack
page.

        -----------------  -> START of page (task->stack)
        |               |
        | thread_info   |
        -----------------
        |               |
   ^    ~               ~
   |    ~               ~
   |    |               |
   |    |               | <---- pt_regs used to END here
        -----------------
        | 1 word GUTTER |
        ----------------- -> End of page (START of kernel stack)

This required special "one-off" considerations in low level code.

The root cause is very likely assumption of "empty" SP by the original
ARC kernel hackers, despite ARC700 always been "full" SP.

So finally RIP one word gutter !

Signed-off-by: Vineet Gupta <vgupta@synopsys.com>
2013-06-22 19:23:21 +05:30
Vineet Gupta
bed30976e7 ARC: pt_regs update #0: remove kernel stack canary
This stack slot is going to be used in subsequent commits

Signed-off-by: Vineet Gupta <vgupta@synopsys.com>
2013-06-22 19:23:21 +05:30
Vineet Gupta
3abc944802 ARC: [mm] Make stack/heap Non-executable by default
1. For VM_EXEC based delayed dcache/icache flush, reduces the number of
   flushes.

2. Makes this security feature ON by default rather than OFF before.

3. Applications can use mprotect() to selectively override this.

4. ELF binaries have a GNU_STACK segment which can easily override the
   kernel default permissions.
   For nested-functions/trampolines, gcc already auto-enables executable
   stack in elf. Others needing this can use -Wl,-z,execstack option.

Signed-off-by: Vineet Gupta <vgupta@synopsys.com>
2013-06-22 19:23:20 +05:30
Vineet Gupta
2ed21dae02 ARC: [mm] Assume pagecache page dirty by default
Similar to ARM/SH

Signed-off-by: Vineet Gupta <vgupta@synopsys.com>
2013-06-22 19:23:19 +05:30
Vineet Gupta
3049918660 ARC: cache detection code bitrot
* Number of (i|d)cache ways can be retrieved from BCRs and hence no need
  to cross check with with built-in constants
* Use of IS_ENABLED() to check for a Kconfig option
* is_not_cache_aligned() not used anymore

Signed-off-by: Vineet Gupta <vgupta@synopsys.com>
2013-06-22 13:46:43 +05:30
Vineet Gupta
da1677b02d ARC: Disintegrate arcregs.h
* Move the various sub-system defines/types into relevant files/functions
  (reduces compilation time)

* move CPU specific stuff out of asm/tlb.h into asm/mmu.h

Signed-off-by: Vineet Gupta <vgupta@synopsys.com>
2013-06-22 13:46:42 +05:30
Vineet Gupta
8235703e10 ARC: Use kconfig helper IS_ENABLED() to get rid of defines.h
Signed-off-by: Vineet Gupta <vgupta@synopsys.com>
2013-06-22 13:46:42 +05:30
Vineet Gupta
7bb66f6e6e ARC: lazy dcache flush broke gdb in non-aliasing configs
gdbserver inserting a breakpoint ends up calling copy_user_page() for a
code page. The generic version of which (non-aliasing config) didn't set
the PG_arch_1 bit hence update_mmu_cache() didn't sync dcache/icache for
corresponding dynamic loader code page - causing garbade to be executed.

So now aliasing versions of copy_user_highpage()/clear_page() are made
default. There is no significant overhead since all of special alias
handling code is compiled out for non-aliasing build

Signed-off-by: Vineet Gupta <vgupta@synopsys.com>
2013-05-25 14:15:55 +05:30
Vineet Gupta
006dfb3c9c ARC: Use enough bits for determining page's cache color
The current code uses 2 bits for determining page's dcache color, thus
sorting pages into 4 bins, whereas the aliasing dcache really has 2 bins
(8k page, 64k dcache - 4 way-set-assoc).
This can cause extraneous flushes - e.g. color 0 and 2.

Signed-off-by: Vineet Gupta <vgupta@synopsys.com>
2013-05-23 14:25:09 +05:30
Vineet Gupta
3e87974dec ARC: Brown paper bag bug in macro for checking cache color
The VM_EXEC check in update_mmu_cache() was getting optimized away
because of a stupid error in definition of macro addr_not_cache_congruent()

The intention was to have the equivalent of following:

	if (a || (1 ? b : 0))

but we ended up with following:

	if (a || 1 ? b : 0)

And because precedence of '||' is more that that of '?', gcc was optimizing
away evaluation of <a>

Nasty Repercussions:
1. For non-aliasing configs it would mean some extraneous dcache flushes
   for non-code pages if U/K mappings were not congruent.
2. For aliasing config, some needed dcache flush for code pages might
   be missed if U/K mappings were congruent.

Signed-off-by: Vineet Gupta <vgupta@synopsys.com>
2013-05-23 14:24:52 +05:30
Vineet Gupta
a950549c67 ARC: copy_(to|from)_user() to honor usermode-access permissions
This manifested as grep failing psuedo-randomly:

-------------->8---------------------
[ARCLinux]$ ip address show lo | grep inet
[ARCLinux]$ ip address show lo | grep inet
[ARCLinux]$ ip address show lo | grep inet
[ARCLinux]$
[ARCLinux]$ ip address show lo | grep inet
    inet 127.0.0.1/8 scope host lo
-------------->8---------------------

ARC700 MMU provides fully orthogonal permission bits per page:
Ur, Uw, Ux, Kr, Kw, Kx

The user mode page permission templates used to have all Kernel mode
access bits enabled.
This caused a tricky race condition observed with uClibc buffered file
read and UNIX pipes.

1. Read access to an anon mapped page in libc .bss: write-protected
   zero_page mapped: TLB Entry installed with Ur + K[rwx]

2. grep calls libc:getc() -> buffered read layer calls read(2) with the
   internal read buffer in same .bss page.
   The read() call is on STDIN which has been redirected to a pipe.
   read(2) => sys_read() => pipe_read() => copy_to_user()

3. Since page has Kernel-write permission (despite being user-mode
   write-protected), copy_to_user() suceeds w/o taking a MMU TLB-Miss
   Exception (page-fault for ARC). core-MM is unaware that kernel
   erroneously wrote to the reserved read-only zero-page (BUG #1)

4. Control returns to userspace which now does a write to same .bss page
   Since Linux MM is not aware that page has been modified by kernel, it
   simply reassigns a new writable zero-init page to mapping, loosing the
   prior write by kernel - effectively zero'ing out the libc read buffer
   under the hood - hence grep doesn't see right data (BUG #2)

The fix is to make all kernel-mode access permissions mirror the
user-mode ones. Note that the kernel still has full access to pages,
when accessed directly (w/o MMU) - this fix ensures that kernel-mode
access in copy_to_from() path uses the same faulting access model as for
pure user accesses to keep MM fully aware of page state.

The issue is peudo-random because it only shows up if the TLB entry
installed in #1 is present at the time of #3. If it is evicted out, due
to TLB pressure or some-such, then copy_to_user() does take a TLB Miss
Exception, with a routine write-to-anon COW processing installing a
fresh page for kernel writes and also usable as it is in userspace.

Further the issue was dormant for so long as it depends on where the
libc internal read buffer (in .bss) is mapped at runtime.
If it happens to reside in file-backed data mapping of libc (in the
page-aligned slack space trailing the file backed data), loader zero
padding the slack space, does the early cow page replacement, setting
things up at the very beginning itself.

With gcc 4.8 based builds, the libc buffer got pushed out to a real
anon mapping which triggers the issue.

Reported-by: Anton Kolesov <akolesov@synopsys.com>
Cc: <stable@vger.kernel.org> # 3.9
Signed-off-by: Vineet Gupta <vgupta@synopsys.com>
2013-05-23 10:33:03 +05:30
Vineet Gupta
5bba49f539 ARC: [mm] Aliasing VIPT dcache support 4/4
Enforce congruency of userspace shared mappings

Signed-off-by: Vineet Gupta <vgupta@synopsys.com>
2013-05-09 22:00:57 +05:30
Vineet Gupta
de2a852cc0 ARC: [mm] Aliasing VIPT dcache support 3/4
Fix the one zillion warnings

Signed-off-by: Vineet Gupta <vgupta@synopsys.com>
2013-05-09 22:00:57 +05:30
Vineet Gupta
4102b53392 ARC: [mm] Aliasing VIPT dcache support 2/4
This is the meat of the series which prevents any dcache alias creation
by always keeping the U and K mapping of a page congruent.
If a mapping already exists, and other tries to access the page, prev
one is flushed to physical page (wback+inv)

Essentially flush_dcache_page()/copy_user_highpage() create K-mapping
of a page, but try to defer flushing, unless U-mapping exist.
When page is actually mapped to userspace, update_mmu_cache() flushes
the K-mapping (in certain cases this can be optimised out)

Additonally flush_cache_mm(), flush_cache_range(), flush_cache_page()
handle the puring of stale userspace mappings on exit/munmap...

flush_anon_page() handles the existing U-mapping for anon page before
kernel reads it via the GUP path.

Note that while not complete, this is enough to boot a simple
dynamically linked Busybox based rootfs

Signed-off-by: Vineet Gupta <vgupta@synopsys.com>
2013-05-09 21:59:46 +05:30
Vineet Gupta
6ec18a81b2 ARC: [mm] Aliasing VIPT dcache support 1/4
This preps the low level dcache flush helpers to take vaddr argument in
addition to the existing paddr to properly flush the VIPT dcache

Signed-off-by: Vineet Gupta <vgupta@synopsys.com>
2013-05-09 21:53:16 +05:30
Vineet Gupta
eacd0e950d ARC: [mm] Lazy D-cache flush (non aliasing VIPT)
flush_dcache_page( ) is MM hook to ensure that a page has consistent
views between kernel and userspace. Thus it is called when

* kernel writes to a page which at some later point could get mapped to
  userspace (so kernel mapping needs to be flushed-n-inv)
* kernel is about to read from a page with possible userspace mappings
  (so userspace mappings needs to be made coherent with kernel ones)

However for Non aliasing VIPT dcache, any userspace mapping will always
be congruent to kernel mapping. Thus d-cache need need not be flushed at
all (or delayed indefinitely).

The only reason it does need to be flushed is when mapping code pages.
Since icache doesn't snoop dcache, those dirty dcache lines need to be
written back to memory and icache line invalidated so that icache lines
fetch will get the right data.

Decent gains on LMBench fork/exec/sh and File I/O micro-benchmarks.

(1) FPGA @ 80 MHZ

Processor, Processes - times in microseconds - smaller is better
------------------------------------------------------------------------------
Host                 OS  Mhz null null      open slct sig  sig  fork exec sh
                             call  I/O stat clos TCP  inst hndl proc proc proc
--------- ------------- ---- ---- ---- ---- ---- ---- ---- ---- ---- ---- ----
3.9-rc6-a Linux 3.9.0-r   80 4.79 8.72 66.7 116. 239. 8.39 30.4 4798 14.K 34.K
3.9-rc6-b Linux 3.9.0-r   80 4.79 8.62 65.4 111. 239. 8.35 29.0 3995 12.K 30.K
3.9-rc7-c Linux 3.9.0-r   80 4.79 9.00 66.1 106. 239. 8.61 30.4 2858 10.K 24.K
                                                                ^^^^ ^^^^ ^^^

File & VM system latencies in microseconds - smaller is better
-------------------------------------------------------------------------------
Host                 OS   0K File      10K File     Mmap    Prot   Page 100fd
                        Create Delete Create Delete Latency Fault  Fault selct
--------- ------------- ------ ------ ------ ------ ------- ----- ------- -----
3.9-rc6-a Linux 3.9.0-r  317.8  204.2 1122.3  375.1 3522.0 4.288     20.7 126.8
3.9-rc6-b Linux 3.9.0-r  298.7  223.0 1141.6  367.8 3531.0 4.866     20.9 126.4
3.9-rc7-c Linux 3.9.0-r  278.4  179.2  862.1  339.3 3705.0 3.223     20.3 126.6
                         ^^^^^  ^^^^^  ^^^^^  ^^^^

(2) Customer Silicon @ 500 MHz (166 MHz mem)

------------------------------------------------------------------------------
Host                 OS  Mhz null null      open slct sig  sig  fork exec sh
                             call  I/O stat clos TCP  inst hndl proc proc proc
--------- ------------- ---- ---- ---- ---- ---- ---- ---- ---- ---- ---- ----
abilis-ba Linux 3.9.0-r  497 0.71 1.38 4.58 12.0 35.5 1.40 3.89 2070 5525 13.K
abilis-ca Linux 3.9.0-r  497 0.71 1.40 4.61 11.8 35.6 1.37 3.92 1411 4317 10.K
                                                                ^^^^ ^^^^ ^^^

Signed-off-by: Vineet Gupta <vgupta@synopsys.com>
2013-05-07 19:08:15 +05:30
Vineet Gupta
94bad1afee ARC: [mm] consolidate icache/dcache sync code
Now that we have same helper used for all icache invalidates (i.e.
vaddr+paddr based exact line invalidate), consolidate the open coded
calls into one place.

Also rename flush_icache_range_vaddr => __sync_icache_dcache

Signed-off-by: Vineet Gupta <vgupta@synopsys.com>
2013-05-07 19:08:13 +05:30
Vineet Gupta
24603fdd19 ARC: [mm] optimise icache flush for user mappings
ARC icache doesn't snoop dcache thus executable pages need to be made
coherent before mapping into userspace in flush_icache_page().

However ARC700 CDU (hardware cache flush module) requires both vaddr
(index in cache) as well as paddr (tag match) to correctly identify a
line in the VIPT cache. A typical ARC700 SoC has aliasing icache, thus
the paddr only based flush_icache_page() API couldn't be implemented
efficiently. It had to loop thru all possible alias indexes and perform
the invalidate operation (ofcourse the cache op would only succeed at
the index(es) where tag matches - typically only 1, but the cost of
visiting all the cache-bins needs to paid nevertheless).

Turns out however that the vaddr (along with paddr) is available in
update_mmu_cache() hence better suits ARC icache flush semantics.
With both vaddr+paddr, exactly one flush operation per line is done.

Signed-off-by: Vineet Gupta <vgupta@synopsys.com>
2013-05-07 19:08:12 +05:30
Vineet Gupta
8d56bec2f2 ARC: [mm] optimize needless full mm TLB flush on munmap
munmap ends up calling tlb_flush() which for ARC was flushing the entire
TLB unconditionally (by moving the MMU to a new ASID)

do_munmap
  unmap_region
    unmap_vmas
      unmap_single_vma
         unmap_page_range
            tlb_start_vma
            zap_pud_range
            tlb_end_vma()
  tlb_finish_mmu
    tlb_flush()  ---> unconditional flush_tlb_mm()

So even a single page munmap, a frequent operation when uClibc dynamic
linker (ldso) is loading the dependent shared libraries, would move the
the ASID multiple times - needlessly invalidating the pre-faulted TLB
entries (and increasing the rate of ASID wraparound + full TLB flush).

This is now optimised to only be called if tlb->full_mm (which means
for exit/execve) cases only. And for those cases, flush_tlb_mm() is
already optimised to be a no-op for mm->mm_users == 0.

So essentially there are no mmore full mm flushes - except for fork which
anyhow needs it for properly COW'ing parent address space.

munmap now needs to do TLB range flush, which is implemented with
tlb_end_vma()

Results
-------
1. ASID now consistenly moves by 4 during a simple ls (as opposed to 5 or
   7 before).

2. LMBench microbenchmark also shows improvements

Basic system parameters
------------------------------------------------------------------------------
Host                 OS Description              Mhz  tlb  cache  mem scal
                                                     pages line   par load
                                                           bytes
--------- ------------- ----------------------- ---- ----- ----- ------ ----
3.9-rc5-0 Linux 3.9.0-r 3.9-rc5-0404-gcc-4.4-ba   80     8    64 1.1000 1
3.9-rc5-0 Linux 3.9.0-r 3.9-rc5-0405-avoid-full   80     8    64 1.1200 1

Processor, Processes - times in microseconds - smaller is better
------------------------------------------------------------------------------
Host                 OS  Mhz null null      open slct sig  sig  fork exec sh
                             call  I/O stat clos TCP  inst hndl proc proc proc
--------- ------------- ---- ---- ---- ---- ---- ---- ---- ---- ---- ---- ----
3.9-rc5-0 Linux 3.9.0-r   80 4.81 8.69 68.6 118. 239. 8.53 31.6 4839 13.K 34.K
3.9-rc5-0 Linux 3.9.0-r   80 4.46 8.36 53.8 91.3 223. 8.12 24.2 4725 13.K 33.K

File & VM system latencies in microseconds - smaller is better
-------------------------------------------------------------------------------
Host                 OS   0K File      10K File     Mmap    Prot   Page 100fd
                        Create Delete Create Delete Latency Fault  Fault selct
--------- ------------- ------ ------ ------ ------ ------- ----- ------- -----
3.9-rc5-0 Linux 3.9.0-r  314.7  223.2 1054.9  390.2  3615.0 1.590 20.1 126.6
3.9-rc5-0 Linux 3.9.0-r  265.8  183.8 1014.2  314.1  3193.0 6.910 18.8 110.4

Signed-off-by: Vineet Gupta <vgupta@synopsys.com>
2013-05-07 13:44:00 +05:30
Christian Ruppert
072eb69390 ARC: [TB10x] Add support for TB10x platform
Infrastructure required to make the Linux kernel compile and boot on the
Abilis Systems TB10x series of SOCs based on ARC700 CPUs:
  - Kmake related files (Kconfig, Makefile, tb10x_defconfig)
  - TB10x platform initialisation

Signed-off-by: Christian Ruppert <christian.ruppert@abilis.com>
Signed-off-by: Pierrick Hascoet <pierrick.hascoet@abilis.com>
Signed-off-by: Vineet Gupta <vgupta@synopsys.com>
2013-05-07 13:43:59 +05:30